Estimating Accuracy of Judgment Using Recorded Interviews

THERE ARE several sources of error in evaluations based on the psychiatric interview, two of these being variations in interviewing techniques and variations in methods of recording observations. The control of these sources of error is a crucial problem in psychiatric research; the solutions offered have included restricting the examiner to a structured interview and having him record his judgments on standardized inventories or rating scales. Even with these restrictions, however, errors due to failure to observe significant behavior and error in judgment itself may remain. For many research undertakings in psychiatry, the usual kind of reliability study—where all of the observers involved in the study make simultaneous but independent judgments of a series of patient interviews—is quite adequate to determine the magnitude of these errors. However, there are many instances where this kind of reliability study cannot be executed. For example, when