Deep Learning-Based Anomaly Detection in Network Intrusion Detection Systems: A Comparative Evaluation

Network intrusion detection systems are fundamental components of organizational cybersecurity infrastructure, responsible for identifying unauthorized access attempts and malicious activities within computer networks. This paper presents a comprehensive comparative evaluation of three deep learning architectures for anomaly-based network intrusion detection: Variational Autoencoders (VAEs), VAE-GANs, and Adversarial Autoencoders (AAEs). Each architecture is evaluated across multiple benchmark datasets including CICIDS2017, UNSW-NB15, and CIC-DDoS2019, using standardized evaluation metrics encompassing detection accuracy, false positive rate, computational efficiency, and robustness against adversarial attacks. We investigate the impact of latent space dimensionality, reconstruction error thresholds, and training data composition on detection performance. Experimental results demonstrate that VAE-GANs achieve the highest detection accuracy (97.2%) on known attack categories, while AAEs exhibit superior performance on zero-day attack detection (91.4%) and adversarial robustness. VAEs offer the best trade-off between accuracy and computational efficiency, making them suitable for resource-constrained deployment scenarios. We provide practical guidelines for deploying deep learning-based NIDS in production environments based on our comprehensive evaluation.