Psychometrics versus Datametrics: Comment on Vacha-Haase’s “Reliability Generalization” Method and Some Epm Editorial Policies

The present article reviews issues regarding test reliability, which is psychometric terminology, and score reliability, which is score-centric terminology. These issues have arisen, in part, due to some EPM editorial policies and Vacha-Haase’s “reliability generalization” proposal. The article includes (a) a brief historical review of reliability terminology, (b) discussion on the emergence of datametrics (loosely defined as the application of psychometry to scores as opposed to an instrument) including a review of textbook authors’uses of psychometric versus datametric terminology, (c) discussion of problems with datametrics, and (d) a critique of Vacha-Haase’s proposed meta-analytic reliability generalization via dummy-coded regression. The article concludes with a brief summary that presents several suggestions.