Self-evolving cyber defense: an analytical review of AI-driven autonomous and adversarial systems

Abstract The growing complexity, size, and dynamism of cyber threats have revealed inherent weaknesses of the traditional, static cybersecurity models. Attackers nowadays take advantage of artificial intelligence, automation, and adversarial learning methods to avoid detection, create new variants of attacks, and maintain long-term intrusions. In response, cybersecurity research has turned to self-evolving cyber defense systems that can engage in constant learning, autonomous decision-making, and co-evolution with intelligent attackers. This review includes a systematic synthesis of the studies on self-evolving cyber defense with a focus on the merging of artificial intelligence, autonomy, and adversarial learning. We discuss the evolution of cyber threats, machine learning and deep learning approaches for adaptive threat detection, as well as autonomous defense systems enabled by reinforcement learning and multi-agent systems. The review also explores the adversarial machine learning as a source of emerging threats and a powerful defense foundation with focus on the co-evolution of the attackers and the defenders. In addition to algorithmic views, the paper has provided an overview of system architectures, evaluation measures, ethics and legal aspects, and real-life implementation of industrial applications in critical infrastructures, military systems, financial services, smart cities, and cyber-physical environments. The major issues concerning scalability, resistance to adaptive opponents, lack of information, and the interaction of humans and AI are addressed. Lastly, the review presents directions of future research, such as entirely autonomous defense ecosystems, hybrid neuro-symbolic systems, quantum-resilient AI security, and intelligence sharing across domains. This work brings together dispersed research in various fields to offer a reference and roadmap on how to progress to the next generation of self-evolving cyber defense systems.