TurboGuard: Discrete latent-space geometric filtering for robust network intrusion detection under evasion attacks

Machine learning–based Network Intrusion Detection Systems (NIDS) are vulnerable to adversarial evasion attacks, where malicious traffic is intentionally manipulated to be misclassified as benign. Existing defenses, including adversarial training and input transformation, can improve robustness but often require attack-specific retraining, incur additional computational cost, or introduce false-alarm trade-offs. This paper presents TurboGuard , a multi-stage adversarial defense framework for robust flow-based NIDS. TurboGuard models legitimate traffic as a benign discrete representation using a Vector Quantized–Variational Autoencoder (VQ-VAE) trained exclusively on benign flows. Rather than relying on non-differentiability as a standalone defense, TurboGuard extracts a six-dimensional geometric evidence profile from the learned latent representation, capturing reconstruction deviation, code-assignment abnormality, local density variation, Voronoi-boundary proximity, distance entropy, and topological displacement. This evidence is integrated through a dual-path architecture that combines Isolation Forest rejection and residual-aware Augmented DNN classification, with both decision thresholds jointly calibrated on disjoint benign data to satisfy a target system-level false positive rate (FPR). TurboGuard is evaluated under a leakage-aware protocol on CIC-IDS2018 and CIC-IDS2017 using FGSM, PGD, DeepFool, and Carlini–Wagner (C&W) attacks, together with full-cascade BPDA/STE and finite-budget score- and hard-label-query attacks. At the 1.0% target-FPR operating point, TurboGuard achieves final-test FPR/TPR pairs of 1.01%/94.86% on CIC-IDS2018 and 1.00%/93.09% on CIC-IDS2017. Under the evaluated non-adaptive attacks, the minimum evasion detection rates are 58.34% and 64.66%, respectively, while additional BPDA/STE and query-based evaluations show that detection decreases under stronger adaptive access. These findings indicate that discrete latent geometry can provide useful security evidence for adversarially robust NIDS, while also highlighting the importance of deployment-specific threshold calibration and the attack-dependent nature of robustness in normalized aggregate-feature space.