In-test effort in PISA achievement: it makes a difference for students and schools, but not countries

Abstract Background Across multiple theories of psycho-educational development, task-specific effort, such as in-test effort, is postulated to be the most closely associated with achievement. Despite this, research has most frequently considered effort from general academic and domain-specific perspectives and limited research has used multilevel approaches. Such research is needed to accurately identify the extent to which task-specific effort, as perceived by students, is associated with academic performance and at what level of analysis (student, school, country) task-specific effort is a viable target for intervention. Such approaches are also needed in order to account for the role of relevant covariates and factors that may inform in-test effort at different levels. Methods Using three-level doubly latent multilevel modelling, this study leverages the 2022 PISA survey and achievement data from 36 OECD countries to assess the extent to which self-reported in-test effort is associated with achievement at the student-, school-, and country-level. This study tests these associations after accounting for key covariates (gender, language background, SES, and messaging about effort). Results In-test effort was significantly associated with achievement at the student- and school-level, but not at the country-level. This suggests that student-reported individual effort and school climates of in-test effort are relevant to consider for student- and school-level achievement, but in-test effort does not explain country-level differences in achievement. SES was also significantly associated with achievement at all levels. Conclusions These findings have implications for practice and policy. Regarding practice, the findings indicate the importance of adopting individually focused and school-wide practices that help students to understand both how they perceive their level of effort and the importance of in-test effort. The findings also indicate that providing students the opportunity to practice applying effort appropriately in independent learning and studying may be worthwhile. Regarding policy, the findings suggest that initiatives related to effort on international assessments would be misguided, and that policy focused on economic equity may be more beneficial.