A Cognitive Intelligence-Based NetBot-AI Model for Proactive Mitigation of Security Threats in Next-Generation VOIP Communication Networks

Voice over IP (VoIP) has become a cornerstone of modern digital communication, yet its integration with 5G, cloud, and satellite infrastructures has expanded the attack surface, exposing critical vulnerabilities to threats such as DoS, eavesdropping, man-in-the-middle intrusions, and zero-day exploits. To address these challenges, this study proposes NetBot-AI, a cognitive intelligence-driven security tool designed for real-time, behaviour-based detection and autonomous mitigation of VoIP threats, including encrypted and adversarial attacks, without reliance on static signatures. The system integrates adaptive Deep Q-Learning with a novel Ekolama Loss Function to stabilise training under high-latency, noisy conditions typical of satellite networks. Using a hybrid Layer 3 and Layer 7 monitoring approach, NetBot-AI was trained on a consolidated dataset of 10 million records, comprising simulated traffic and live satellite-linked VoIP calls, spanning eight threat categories and multiple protocols (SIP, RTP, H.323, MGCP). Empirical results demonstrate exceptional performance: 97–99% binary classification accuracy, 94–98% multi-class accuracy, and 99.92% reliability via max-voting ensemble. Crucially, NetBot-AI achieved 97.50% detection accuracy, 96.25% mitigation accuracy, and 96.85% F1-score, substantially outperforming existing tools by over 29 percentage points. Deployed via a WS-Security–enabled SOAP API, NetBot-AI ensures interoperability with legacy infrastructures while enabling proactive, context-aware defence. This work establishes a new benchmark for intelligent, scalable, and robust VoIP security in heterogeneous, next-generation network environments.