Templatic Morphology as Decodable Geometry, and Abjad Letter-Values as an Attention Probe, in Classical Arabic Transformer Models

Key Points

• The study aims to explore how templatic morphology and letter-value structure are processed in Arabic and Hebrew transformer models.
• Conducted a pre-registered cross-linguistic study using classical-Arabic and classical-Hebrew transformer encoders (CAMeLBERT-ca, AlephBERT).
• Analyzed 2,322,977 cross-root word pairs for mutual attention based on Abjad values.
• Utilized permutation tests to evaluate significance against a frequency-matched null analysis.
• The Arabic morphological pattern (wazn) is represented as a geometrically separable, linearly decodable structure.
• Pairs with proximate Abjad values showed higher mutual attention (partial ρ = −0.0558, p = 0.026).
• The Hebrew control confirmed the general findings, while frequency alignment specific to Mashriqi was identified.