Federated Deep Reinforcement Learning for Efficient Jamming Attack Mitigation in O-RAN

Open RAN (ORAN or O-RAN) revolutionizes Radio Access Networks (RAN) by offering flexibility and cost-efficiency through inter-vendor equipment interoperability. More importantly, it addresses emerging security threats, such as jamming attacks, by incorporating network softwarization and leveraging Artificial Intelligence (AI) techniques. However, AI-based systems face challenges such as limited training data, slow convergence, and vulnerability to dynamic attack patterns like Zero-day attacks. To enhance jamming attack mitigation in O-RAN, Multi-Agent Reinforcement Learning (MARL) has been introduced for improved flexibility and robustness. However, MARL requires data sharing, which consumes network bandwidth and slows down training, and the curse of dimensionality limits its benefits due to the exponential growth of the state-action space. To overcome these limitations, we provide a novel framework that combines federated learning (FL) and deep reinforcement learning (DRL) for efficient jamming attack detection in O-RAN. FL allows decentralized agents to train local models using their data sources, and the models are aggregated into a global model at a Non-real-time RAN Intelligent Controller (RIC) to guide decision-making. The federated learning process enables distributed intelligence, while deep reinforcement learning ensures adaptive and robust jamming attack detection. Our proposed framework improves security, privacy, and resilience in ORAN through collaborative FL and adaptive DRL. Extensive simulations demonstrate its superiority in detection accuracy, resource efficiency, and scalability.