Political bias in transformer-based language models poses a critical challenge for applications involving politically sensitive Arabic news, yet systematic evaluation remains limited. This paper presents a multi-view behavioral framework to detect political bias in four pre-trained transformer models: AraBERTv2, CAMeLBERT, mBERT, and XLM-R. The framework integrates four complementary probes: sentiment drift, emotion drift, counterfactual actor-swapping for identity sensitivity, and masked language model probing to detect lexical preference shifts. Each model is evaluated before and after domain-adaptive fine-tuning on the FigNews Arabic political news dataset to analyze how politically sensitive training data influences representational bias. To synthesize signals from these probes, a Decision and Bias Reporting Agent (DBRA) aggregates the evidence using a structured hierarchy that prioritizes implicit bias indicators. Results show that bias is already present in base checkpoints and can significantly shift after adaptation. For example, mBERT’s masked preference for SideA drops from 40.7% to 0.0%, indicating complete directional collapse, while XLM-R shows a large increase in masked preference toward SideA (ΔPR = +32.8%).
Abdelhameed et al. (Thu,) studied this question.