Comparative Analysis of Transformer-Based Models for Question Answering on SQuAD v2.0

Transformer-based architectures have revolutionized Question Answering (QA), yet navigating the trade-off between architectural complexity, computational efficiency, and robust generalization remains a critical challenge. This study presents a rigorous, side-by-side evaluation of four distinct Transformer models—ALBERT, RoBERTa, DeBERTa, and XLNet—on the SQuAD v2.0 benchmark, which demands both precise span extraction and calibration to abstain from unanswerable queries. By employing identical preprocessing pipelines and hyperparameter schedules, we isolate the impact of specific architectural innovations, including dynamic masking, disentangled attention, and permutation language modeling. Experimental results establish RoBERTa as the performance leader (85.90% EM, 88.69% F1), validating the efficacy of robust optimization strategies. Notably, DeBERTa followed closely (85.41% EM, 88.54% F1) but achieved superior performance on unanswerable questions (88.24%), suggesting that its disentangled attention mechanism offers enhanced verification capabilities against distractor entities. XLNet demonstrated competitive potential (78.30% EM) via its autoregressive formulation, while ALBERT confirmed its status as the efficiency benchmark, delivering respectable accuracy (80.70% EM) with a fraction of the parameter footprint. Qualitative error analysis reveals that while lighter models are prone to "hallucinating" answers, advanced architectures primarily fail due to subtle reasoning errors rather than entity mismatch. These findings provide a definitive decision framework for practitioners: prioritizing RoBERTa and DeBERTa for maximum reliability in complex domains, and leveraging ALBERT for scalable, resource-constrained deployments.