AIOps for Autonomous Incident Detection and Causal Root-Cause Analysis in Kubernetes Clusters

Hyper-scale cloud-native applications deployed on Kubernetes comprise thousands of ephemeral microservice pods interconnected through dynamic service meshes, distributed caches, and asynchronous message brokers. When a localized infrastructure degradation or bad canary rollout occurs, retry amplification and synchronous remote procedure call (RPC) blocking propagate latency spikes upstream within milliseconds, triggering alert storms that overwhelm Site Reliability Engineering (SRE) on-call teams. Conventional threshold alerting and pairwise Pearson correlation engines fail to distinguish upstream symptomatic victims from the true originating root cause, resulting in prolonged Mean Time to Detect (MTTD) and Mean Time to Resolve (MTTR). This paper presents an end-to-end Artificial Intelligence for IT Operations (AIOps) architecture for autonomous incident detection and causal Root-Cause Analysis (RCA) in Kubernetes clusters. Our system ingests zero-instrumentation Linux kernel eBPF socket and syscall telemetry alongside OpenTelemetry metrics, logs, and distributed traces into a unified spatio-temporal representation. A Variational Graph Autoencoder (VGAE) coupled with Drain3 log template extraction detects multi-dimensional anomalies within 15 seconds of onset. To isolate the true culprit pod or node resource, we combine the Peter-Clark (PC) conditional independence algorithm with interventional Do-calculus and Personalized PageRank over the live Kubernetes deployment topology. We examine production case studies from the Netflix Titus container orchestration platform and Uber's uMonitor and CauseInfer microservice RCA engines. Empirical evaluations across a 120-service Kubernetes chaos testbed (3,600 pods) under 420 injected fault scenarios demonstrate a Top-1 root-cause localization accuracy (AC@1) of 86.4% and Top-3 accuracy (AC@3) of 95.8%, reducing alert volume by 93.8%, cutting MTTD from 8.40 minutes to 0.42 minutes, and lowering MTTR from 47.50 minutes to 4.15 minutes. Research Domain & Specialization Track Master of Science in Information Technology (MSIT) — Site Reliability Engineering and Cloud Observability. Core Technical Highlights Empirical Production Case Studies: Rigorous architectural blueprints and telemetry evaluation across enterprise cloud deployments. Four-Tier Reference Architecture: Formal specification from edge ingress to distributed hardware execution pools. Mathematical Formulation: Formally derived algorithmic and optimization models with closed-loop verification. Multi-Modal Benchmarking: Comparative empirical baselines under fault-injection and synthetic workload scaling. Author Information Author: Sonu Kumar SinghRole: Senior Consultant Affiliation: Independent Researcher Publication Status: Preprint / Working Paper (MSIT AI & Cloud Systems Research Series) Reproducibility & Open Science All included architecture diagrams, telemetry schemas, and benchmark curves are open-access under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.