Applied Explainability for Large Language Models: A Comparative Study

Large Language Models (LLMs) achieve strong performance across natural language processing tasks, yet their internal decision processes remain difficult to interpret. This lack of transparency creates challenges in real-world deployments requiring trust, debugging, and accountability. This study presents a comparative analysis of three explainability techniques—Integrated Gradients, Attention Rollout, and SHAP—applied to a fine-tuned DistilBERT model on the SST-2 sentiment classification task. The methods are evaluated under a consistent experimental setup using qualitative criteria such as faithfulness, stability, and interpretability. The results show that gradient-based attribution methods provide the most stable and intuitive explanations, while attention-based approaches are computationally efficient but less aligned with prediction-relevant features. Model-agnostic methods offer flexibility but introduce computational overhead and variability. This work highlights practical trade-offs in explainability techniques and emphasizes the importance of evaluating them in realistic scenarios. The findings provide actionable insights for machine learning practitioners working with transformer-based NLP systems.