Effects of Average Signed Area Between Two Item Characteristic Curves and Test Purification Procedures on the DIF Detection via the Mantel-Haenszel Method

Abstract In this study we investigated the effects of the average signed area (ASA) between the item characteristic curves of the reference and focal groups and three test purification procedures on the uniform differential item functioning (DIF) detection via the Mantel-Haenszel (M-H) method through Monte Carlo simulations. The results showed that ASA, rather than the percentage of DIF items in the test, determines the performances of the conventional one-stage M-H method. For example, the M-H method performed appropriately even when there were 50% DIF items in the test, as long as ASA approaches 0. Under most of the simulated conditions, the two-stage and iterative M-H methods performed much superior to the one-stage M-H method. When tests were short and the mean abilities of the reference and focal groups were far apart, all the three M-H methods yielded inflated Type I error under the two-parameter or three-parameter logistic model. For most of the other situations, the two-stage and iterative M-H methods can be used safely.