Federated Reinforcement Learning for Energy-Efficient D2D-IoT Networks with AoI Awareness

This paper proposes a federated reinforcement learning (FRL) framework for optimizing energy efficiency (EE) and Age of Information (AoI) in device-to-device (D2D) and Internet of Things (IoT) networks. The model leverages simultaneous wireless information and power transfer (SWIPT) with heterogeneous energy harvesting mechanisms-time switching (TS) for D2D users and power splitting (PS) for IoT devices. The objective is to maximize EE while satisfying constraints on data rate, AoI, power transmission, spectrum sharing, and time allocation. The resulting non-convex mixed-integer nonlinear programming problem is addressed using an FRL approach, where a software-defined network controller coordinates distributed agents to optimize resource allocation while preserving data privacy. Simulations demonstrate that the proposed framework achieves up to 25% higher EE and maintains AoI below critical thresholds compared to baseline methods, offering a scalable solution for energy-constrained, time-sensitive communication systems.