Devising maintainable adaptive AI (MAI) model for improved network intrusion detection

Network intrusion detection systems (IDS) are still a staple of the contemporary cybersecurity infrastructure, but the traditional methods based on fixed signatures and theoretical rule sets are still not able to overcome the high false positive rates, slow detection and a general inability to follow the fast changing tactics used by attacks. These are especially acute in settings where encrypted traffic streams, heterogeneous IoT devices and advanced adversarial interactions that strategically bypass signature-based defenses are present. In this paper, the author presents the Maintainable Adaptive AI (MAI) model, a new two-stage ensemble architecture, which combines gradient boosting (XGBoost) with a lightweight feedforward neural network to provide real-time continuously adaptive intrusion detection. The MAI model also uses dynamic feature selection of the lexical, host-based, traffic-based, and statistical attributes in addition to an incremental learning process directed by the KL-divergence-based drift detection. In contrast to earlier, ad hoc proposals of adaptive IDS, the MAI framework is specifically maintainable by a selective retraining of components that only increases the weight of the neural networks when in standard operation and initiates a complete ensemble recalibration when distribution drift is beyond predefined thresholds. Thorough testing using four different sets of data, which are the Malicious URLs Dataset (651,191 entries), ISCX VPN-NonVPN Traffic Dataset (200,000 samples), TON IoT Dataset (400, 000 samples), and a newly curated Custom Scraped Dataset (cumulative of approximately 200,000 real-world attack logs across open-source threat intelligence feeds) prove precision of 93–96, recall of 92–95 and overall Formal complexity analysis The computational tractability of the model is defined and inference latencies of 1.8 ms per sample with batched computation on a GPU, which can be deployed at network links with throughput as high as 4.3 Gbps. Initial adversarial robustness evaluation shows that the model still achieves more than 83% accuracy with combined URL perturbation attacks, and the competitiveness and even better performance are proved by the comparison to other recent state-of-the-art solutions such as federated boosting algorithms, collaborative SRU networks, and concept-drift-sensitive random forests. The experimental code and model settings are accessible in the public to reproduce and research them at the repository where Data Availability section is provided.