Synapse
⌘+K
Synapse
PulseExploreClubsResearchersJournals
Instagram
HomeClubsExplore
October 5, 2025Open Access

Linguistic Interpretability of Transformer-based Language Models: a systematic review

View Full Paper
Ask AI
Bookmark
Share

Authors

MLMiguel López-OtalJGJorge GraciaJBJordi Bernad

Discussion

Loading...

Member takes

Overview

Systematic review analyzes linguistic knowledge in transformer language models, highlighting interpretability challenges.

Key Points

  • Transformer-based language models exhibit excellent performance in various language tasks, yet their internal workings remain unclear, creating a black box scenario.
  • The review encompasses 160 studies across multiple languages and models, aiming to uncover linguistic phenomena such as syntax, morphology, and semantics.
  • The lack of research focusing on linguistic interpretability creates limitations in understanding how these models encode information similar to humans.
  • The analysis emphasizes pre-trained language models and highlights the importance of interpretability techniques in revealing internal representations.

Cite This Study

López-Otal et al. (2025) studied this question.

synapsesocial.com/papers/68e24e59d6d66a53c2472eb4https://doi.org/10.48550/arxiv.2504.08001
View Full Paper
Ask AI
Bookmark
Share

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1A Primer on the Inner Workings of Transformer-based Language Models2024 · 9 citations
  2. 2TRACE: Training and Inference-Time Interpretability Analysis for Language Models2025
  3. 3Representations as Language: An Information-Theoretic Framework for Interpretability2024
  4. 4Explainability for Large Language Models: A Survey2024 · 626 citations
  5. 5Exploring the architectures of large language models and impact across multiple domains: a review2026