Adaptive Federated Learning with Reinforcement Learning-Based Client Selection for Heterogeneous Environments

This study introduces an Adaptive Federated Learning (AFL) framework designed to address the challenges of data heterogeneity, resource imbalance, and communication constraints in decentralized learning environments. The framework integrates reinforcement learning (RL) based client selection using both Tabular Q-Learning and Deep Q-Network (DQN) strategies to dynamically identify clients that most positively impact global model performance. A multi-objective reward function, combining model accuracy and execution time, guides the RL agent toward performance- and efficiency-aware client selection. For local model training, Random Forest (RF) classifiers are employed to ensure robustness to noise, class imbalance, and limited computational resources, particularly in privacy-sensitive healthcare settings. The AFL framework is evaluated on two real-world healthcare datasets BRFSS2015 and Diabetes Prediction, and extended to benchmark FL datasets (CIFAR-10 and FEMNIST) to assess scalability and generalization. Experimental results demonstrate that the DQN-based AFL achieves superior global accuracy (up to 91.3%) compared to Tabular Q-Learning and baseline methods such as FedAvg, while also reducing execution time by up to 15%. Client-level accuracy remains stable across rounds, with reward progression confirming effective RL policy convergence. These findings underscore the AFL framework’s capability to adaptively balance performance and efficiency, offering a practical and scalable solution for federated learning in heterogeneous, resource-constrained environments.