Mechanistic Interpretability (MI), originating from traditional interpretability which tracks the reasons behindneural network behaviors, dedicates itself to reconstructing neural network models into human-understandablealgorithms, often expressible as pseudocode or referred to as circuits. MI typically attributes each step in the pseudocodeto specific model modules, thereby accomplishing a semantic decomposition of the neural network model.This methodology is particularly effective for large models and has thus risen in prominence with the emergenceof large language models, attracting growing attention from the perspectives of AI safety and reliability. However,this field’s rapid growth has led researchers to adopt disparate concepts and methods, resulting in a lack of a unifiedframework. Moreover, the precise meaning of “mechanistic” remains ambiguous, and the distinction between MI andexisting interpretability methods has yet to be clearly established. In this paper, we first survey the historical and culturalbackground of MI, clarify its differences from traditional interpretability approaches, and propose a conceptualframework that organizes key ideas in MI. Additionally, we discuss MI methods and their limitations, ranging fromobservational to interventional approaches. Finally, we explore current challenges in MI research and offer directionsfor future work to understand increasingly complex AI systems and ensure their safety.
Aoki et al. (Wed,) studied this question.