Validity of Approximation Techniques for Detecting Item Bias

The purpose of this research was to recommend an item bias procedure when the number of minority examinees is too small to use preferred three‐parameter IRT methods. The chi‐square, Angoff delta‐plot, andpseudo‐IRT indices were compared with both real and simulated data. For the real test data a criterion of known bias had been established by cross‐validated IRT‐3 results. The findings from the Math Test and the simulated test were consistent. The pseudo‐IRT approach was best (measured by both correlations and percent agreement) in delecting criterion bias. The chi‐square was close in accuracy to the pseudo‐IRT index. The Angoff delta‐plot method was found to be inadequate on both heuristic and empirical grounds. In extreme cases it even identified items as biased against whites that were simulated to be biased against blacks. However, a modified Angoff index, where p‐value differences were regressed on item point biserials (and the residualized values used as the index), was nearly as good as the chi‐square in identifying known bias. A final caution was offered regarding the use of item bias techniques. The statistical flags should never be used mechanically to discard items; rather they should be used to inspect items for possible differences in meaning.