Adaptive Behavioral Zero Trust Frameworks for Identifying Rogue and Compromised Autonomous AI Agents

This article presents the Adaptive Behavioral Zero Trust Framework (AB-ZTF), a novel governance architecture integrating continuous behavioral profiling, Bayesian dynamic trust scoring, and adaptive policy enforcement for real-time identification of rogue and compromised autonomous AI agents. Rapid deployment of autonomous AI agents introduces complex vulnerabilities that conventional Zero Trust Architecture frameworks cannot address, as none integrate continuous behavioral profiling, dynamic trust scoring, and adaptive policy enforcement for agent governance. The framework was evaluated through a three-stage ensemble detection pipeline comprising unsupervised anomaly scoring, sequential LSTM-based behavioral classification, and Temporal Graph Neural Network-based lateral interaction analysis, applied to the CICIDS2017 benchmark dataset with SMOTE-based class balancing across five mapped AI agent threat classes. LightGBM achieved the highest weighted F1-Score of 0.9989 and AUC-ROC of 0.9999, while the Bayesian Trust Engine accurately classified 85.1% of agents, recording a 3.2-step Mean Time to Detection, a False Positive Rate of 0.0031, and a False Negative Rate of 0.0004. Under sustained adversarial feature perturbation, ensemble models maintained F1-Scores above 0.74. Unlike prior behavioral analytics and Zero Trust proposals that treat trust as static at deployment, the AB-ZTF achieves continuous runtime trust updating, surpassing existing benchmarks such as the 0.942 F1-Score reported for Isolation Forest-based non-human entity authentication and representing the first framework to empirically unify behavioral anomaly detection, Bayesian trust scoring, and adaptive Zero Trust policy enforcement for autonomous AI agent governance. These results are obtained using network intrusion data as a structured behavioral proxy, and the framework's direct applicability to live autonomous AI agent telemetry streams remains subject to further validation with agent-native datasets. AB-ZTF provides security layer for enterprise AI operations enforcing adaptive access control across AI ecosystems.