Assessing statistical aspects of test fairness with structural equation modelling

Test fairness and test bias are not synonymous concepts. Test bias refers to statistical evidence that the psychometrics or interpretation of test scores depend on group membership, such as gender or race, when such differences are not expected. A test that is grossly biased may be judged to be unfair, but test fairness concerns the broader, more subjective evaluation of assessment outcomes from perspectives of social justice. Thus, the determination of test fairness is not solely a matter of statistics, but statistical evidence is important when evaluating test fairness. This work introduces the use of the structural equation modelling technique of multiple-group confirmatory factor analysis (MGCFA) to evaluate hypotheses of measurement invariance, or whether a set of observed variables measures the same factors with the same precision over different populations. An example of testing for measurement invariance with MGCFA in an actual, downloadable data set is also demonstrated.