Intersectional Biases Within an Introductory Computing Assessment

Assessments that can measure student understanding of concepts in a reliable and valid way are incredibly valuable in research. Unfortunately, assessments can be a source of bias, differentially impacting students along various demographic lines. Differential Item Functioning (DIF) is a method to explore assessment bias. However, DIF is primarily limited to a single binary demographic variable (e.g. white and non-white; male and female). In this paper, we describe a novel expansion of DIF methods to explore intersections of student identities. We demonstrate the use of classic DIF on a data set of 255 complete responses to a CS1 assessment using binary race and gender variables in our analyses. Then, we present the importance of intersectional DIF by running a similar analysis on intersectional data. Using these methods, we identify problematic items on the assessment that are biased against certain groups of test-takers. Our work contributes an innovative method to help interpret assessment results and inform changes to assessments.