CRFD: Causal and Interpretable Fault Diagnosis Using Counterfactual Reasoning for Microservices on Multi-source Observability Data

Causal and interpretable fault diagnosis is essential for ensuring the stable operation and efficient maintenance of large-scale microservice systems. However, the existing fault diagnosis approaches confront two significant challenges: (1) existing approaches usually suffer from a lack of causality, which leads to their tendency to attribute faults to superficial phenomena rather than investigating the root causes behind superficial phenomena. (2) existing approaches typically suffer from a lack of interpretability, which makes them difficult to provide effective guidance for subsequent troubleshooting. In this study, we propose CRFD, a causal and interpretable fault diagnosis approach based on multi-source observability data for microservice systems, which uses counterfactual reasoning to diagnose faults and provide effective guidance for subsequent troubleshooting. CRFD transforms multi-source observability data into Structural Causal Models (SCMs). Then, it uses a Causal Variational Graph Autoencoder to learn SCMs, and detects anomalies based on SCMs. When an anomaly is detected, counterfactual reasoning is performed based on SCMs to localize root causes and tell operators how to deal with them to effectively mitigate faults. Experimental results on two datasets show that CRFD outperforms all baselines in terms of anomaly detection and root cause localization.