Differential item functioning in PIRLS 2021: An examination of socioeconomic status-based differences in reading comprehension items

Reading comprehension skills are critical predictors of students’ academic performance. This study, based on PIRLS 2021 paper-based test data, investigates whether multiple-choice reading comprehension items function differently according to students’ socioeconomic status (SES) in the samples of Türkiye, Ireland, and South Africa. A total of 21 multiple-choice items drawn from the following passages were analyzed: Where is the Honey?, Pemba Sherpa, and Marie Curie. The analyses of Differential Item Functioning (DIF) were conducted by using Mantel–Haenszel (MH) test, Logistic Regression (LR), and Lord’s Chi-Square method. The findings revealed that the highest number of items exhibiting Differential Item Functioning (DIF) was observed in Turkey, while the lowest was found in South Africa. Additionally, Logistic Regression was identified as the method that detected the greatest number of DIF items. Observed differences in item functioning by SES varied significantly across countries, passages, and DIF detection methods. Notably, the number of DIF items identified in the Turkish sample was relatively high. Furthermore, Logistic Regression (LR) demonstrated higher sensitivity in detecting DIF compared to the other methods employed. The results underscore the importance of evaluating measurement fairness not only from a statistical standpoint but also within contextual, cultural, and cognitive dimensions.