An IRTree approach to investigating examinee-level predictors of disengagement on the PISA

Low test-taking effort remains a persistent problem in low-stakes assessments as it introduces construct-irrelevant variance into the scores which can compromise the validity interpretations made with these scores. This study applies the IRTree Model for Disengagement to jointly estimate disengagement and science ability in the U.S.A. sample of the 2022 Programme for International Student Assessment. We examined how disengagement relates to student-level characteristics, including gender, grade level, whether the language spoken at home is different from that of the test, socioeconomic status, growth mindset, sense of belonging, family support, and experiences of bullying. Results indicated that male students and those in higher grade levels were significantly more likely to disengage. Language background also exhibited a credible effect, with students speaking English at home showing higher disengagement. Socioeconomic status, sense of belonging, and family support showed directional tendencies that provided moderate evidence of association with disengagement, whereas growth mindset, bullying, and educational expectations exhibited no significant effects. A strong negative latent correlation between science ability and disengagement confirmed that lower-performing students were substantially more likely to disengage. The study underscores the importance of accounting for individual differences in test-taking motivation interventions and highlights the utility of IRTree models for producing more valid inferences from low-stakes assessment data.