Key points are not available for this paper at this time.
Research in developmental and educational psychology has come to rely less on conventional psychometric tests and more on records of behavior made by human observers in natural and quasi-natural settings. Three coefficients that purport to reflect the quality of data collected in these observational studies are dis-cussed: the interobserver agreement percentage, the reliability coefficient, and the generalizability coefficient. It is concluded that although high interobserver agreement is desirable in observational studies, high agreement alone is not sufficient to insure the quality of the data that are collected. Evidence of the reliability or generalizability of the data should also be reported. Further ad-vantages of generalizability designs are discussed. Almost everyone engaged in research recog-nizes the need for reliable measuring instru-ments. Reliability is a central topic particu-larly for courses and textbooks concerned with the behavioral sciences. In spite of vary-ing theoretical derivations, its definition is remarkably uniform: A reliable instrument is one with small errors of measurement, one that shows stability, consistency, and depend-ability of scores for individuals on the trait, characteristic, or behavior being assessed. The preparation of this article was supported in
Sandra K. Mitchell (Thu,) studied this question.