Observed-Score Equating With a Heterogeneous Target Population

Test equating is a statistical procedure for adjusting for test form differences in difficulty in a standardized assessment. Equating results are supposed to hold for a specified target population (Kolen & Brennan, 2004 Kolen, M. J. and Brennan, R. J. 2004. Test equating, scaling, and linking , 2nd ed., New York, NY: Springer-Verlag. [Crossref] , [Google Scholar]; von Davier, Holland, & Thayer, 2004 von Davier, A. A., Holland, P. W. and Thayer, D. T. 2004. The kernel method of test equating, New York, NY: Springer-Verlag. [Crossref] , [Google Scholar]) and to be (relatively) independent of the subpopulations from the target population (see Dorans & Holland, 2000 Dorans, N. J. and Holland, P. W. 2000. Population invariance and equitability of tests: Basic theory and the linear case. Journal of Educational Measurement, 37(4): 281–306. [Crossref], [Web of Science ®] , [Google Scholar]; Zumbo, 2007 Zumbo, B. D. 2007. “Validity: Foundational issues and statistical methodology”. In Handbook of statistics, Vol. 26: Psychometrics, Edited by: Rao, C. R. and Sinharay, S. 45–79. Amsterdam, The Netherlands: Elsevier Science B.V. [Google Scholar]). This study discusses the challenges in defining a target population for test equating and in validating the inferences that can be made for the equated scores when the test takers cluster in distinctive ability groups. This discussion takes place in the context of measurement validity (Zumbo, 2007 Zumbo, B. D. 2007. “Validity: Foundational issues and statistical methodology”. In Handbook of statistics, Vol. 26: Psychometrics, Edited by: Rao, C. R. and Sinharay, S. 45–79. Amsterdam, The Netherlands: Elsevier Science B.V. [Google Scholar]) and optimal sampling design (Berger, 1997 Berger, M. P. F. 1997. “Optimal designs for latent variable models: A review”. In Application of latent trait and latent class models in the social sciences, Edited by: Rost, J. and Langeheine, R. 71–79. Muenster, , Germany: Waxmann. [Google Scholar]). This article discusses an alternative observed-score equating (OSE) approach that takes the advantage of the OSE framework described in von Davier (2011 von Davier, A. A. 2011. “A statistical perspective on equating test scores”. In Statistical models for test equating, scaling, and linking, Edited by: von Davier, A. A. 1–17. New York, NY: Springer-Verlag. [Crossref] , [Google Scholar]). The flexibility of the OSE framework and the availability of the standard error of equating difference, which is the standard error of the difference between two equating functions obtained from using two different methods, allow practitioners to compare statistically the equating results from different weighting schemes for distinctive subgroups of the target population. Simulated and real data were used in this study. Item response theory OSE with multigroup calibration, followed by computing the distributions with appropriate sampling weights, was used as the equating criterion.