Enhancing ML Explainability with Multi-Agent LLMs: A Context-Aware XAI Approach

Explainable AI (XAI) techniques enhance ML interpretability but often require technical expertise. We propose a multi-agent architecture that improves LLM-generated explanations through structured reasoning and contextual retrieval. Our system integrates web search, Retrieval-Augmented Generation (RAG), and XAI outputs via specialized agents. Experiments on the Adult dataset show that our approach outperforms standard LLM explanations by 7% in Context Awareness. Additionally, we validate LLM as a Judge, achieving over 80% correlation with human evaluations. Different LLMs, including OpenAI’s GPT-4o and GPT-4o-mini, were tested, highlighting the effectiveness of multi-agent systems in ML explainability.