What question did this study set out to answer?

The aim is to develop a scalable architecture for federated deep reinforcement learning that addresses challenges like non-IID data and convergence issues.

March 30, 2026Open Access

A Scalable Federated Deep Reinforcement Learning Architecture for Collaborative Learning

Key Points

The aim is to develop a scalable architecture for federated deep reinforcement learning that addresses challenges like non-IID data and convergence issues.
Developed a Scalable Federated Deep Reinforcement Learning (SFDRL) architecture.
Introduced performance-aware aggregation of local policy updates.
Implemented selective communication to minimize overhead in distributed systems.
Conducted extensive experiments to evaluate performance and scalability.
Achieved faster convergence and improved stability compared to conventional FedRL methods.
Reduced communication overhead by 28% compared to FedAvg and 64% compared to FedRL.
Obtained higher cumulative rewards and lower variance during training.
Demonstrated effectiveness in large-scale distributed intelligent systems.

Abstract

Federated Learning enables collaborative model training without sharing raw data, while Deep Reinforcement Learning provides powerful mechanisms for sequential decision-making. However, their integration suffers from limited scalability, sensitivity to non-IID data, and unstable convergence in distributed environments. This paper proposes a Scalable Federated Deep Reinforcement Learning (SFDRL) architecture in which distributed agents learn local policies and periodically contribute to a global model via an adaptive, performance-aware aggregation strategy. Unlike conventional FedRL methods that rely on uniform averaging, SFDRL weights local updates according to their learning effectiveness, resulting in faster convergence and improved stability under heterogeneous data distributions. In addition, a selective communication mechanism is introduced to reduce communication overhead by up to 28% and 64% compared with FedAvg and FedRL, respectively. Extensive experiments demonstrate that SFDRL outperforms compared methods, achieving higher cumulative rewards, reduced variance during training, and improved scalability in large-scale distributed settings. These results confirm the suitability of SFDRL for practical deployment in distributed intelligent systems.

A Scalable Federated Deep Reinforcement Learning Architecture for Collaborative Learning

Key Points

Abstract

Cite This Study