An Agentic AI System for Context-Grounded and Deterministic Failure Triage in Data Pipelines

Operational failures in modern data pipelines often require time-consuming manual inspection of large, unstructured logs across heterogeneous systems. We present an agentic, event-driven decision- support framework that transforms raw pipeline failure telemetry into structured incident summaries con- taining representative evidence, root-cause explanations, and remediation guidance for human operators. The system is implemented as loosely coupled services connected via a publish–subscribe broker: (i) telemetry ingestion, (ii) event broker, (iii) anomaly service, (iv) context-grounded RCA reasoning, and (v) notification delivery. A key design choice is deterministic evidence selection prior to automated reasoning: telemetry is aggregated by execution context, noise is suppressed, and a representative failure signal is selected to reduce error propagation into downstream inference. The RCA component constructs a grounded reasoning context using execution artifacts (e.g., ETL scripts and schema representations) and produces structured outputs with recommended and candidate actions; a conservative fallback path is used when automated reasoning fails. We evaluate the framework using offline replay of 100 AWS Glue failure instances with human- labeled ground truth. The system achieves 96% top-1 action accuracy and 97% top-3 hit rate, with 100% safety compliance and 0% hallucination rate. Evidence selection compresses an average of 16.18 ev- idence lines into a single representative anchor per failure, while maintaining 100% anchor precision under a deterministic signal check. End-to-end latency from alert creation to email notification has me- dian 8.0 seconds (p95 20.25 seconds), with 100% RCA generation success and 100% alert-to-notification delivery