An Analysis of DIF and Sources of DIF in Achievement Motivation Items Using Anchoring Vignettes

This study evaluated differential item functioning (DIF) in achievement motivation items before and after using anchoring vignettes as a statistical tool to account for group differences in response styles across gender and ethnicity. We applied the nonparametric scoring of the vignettes to motivation items from the 2015 Programme for International Student Assessment (PISA) and examined changes to DIF characteristics between the raw self-report item scores and the vignette-adjusted item scores. Overall, applying the anchoring vignettes changed DIF classification, magnitude, direction, and form for some items. Group-specific response patterns by gender and by ethnicity group were also seen. Our findings contribute to the research literature on observed response styles in PISA motivation items, DIF in PISA items, and changes to DIF characteristics after adjusting self-report item scores with anchoring vignettes.