Evaluation of Psychotherapy with Alcoholics; A Critical Review

A review of 49 studies published in the United States and Canada from 1952 through 1963 which reported evaluation of psychotherapy with alcoholics was undertaken. These studies fail to live up to their potential for contributing to knowledge in the field because of a failure to meet many methodological requirements for the conduct of evaluative research. Requirements which should be considered in the planning of such research are as follows: 1. Use of controls: The majority of studies used an inadequate design, that of employing the patient as his own control. A few employed another treatment group as a control condition but almost invariably failed to meet the constraint of matching groups or of random assignment of patients to various treatments. The key variable, motivation for treatment, was not controlled for in any study, although two sets of investigations made serious attempts to do so. 2. Subject selection procedures: Although a certain degree of bias is inherent in evaluative research, many of the studies showed severe but easily remediable deficiencies in sampling procedures. In only a few instances were explicit sampling constraints employed to methodological advantage. 3. Selection and definition of criterion variables: The behavior most frequently evaluated was change in drinking pattern, with the ideal goal of abstinence. The question is raised as to whether changes in drinking behavior are measurable, and whether decrease or cessation of drinking is a relevant and adequate measure. Other behaviors selected for measurement suffer from inadequate methodological handling. 4. Measurement instruments and their reliability: The studies reveal a lack of imagination with respect to choice of measurement instruments; the interview served as a major source of data. The concern voiced by some writers over reliability of measurement went largely unheeded in actual practice. 5. Measurement before and after treatment: Collection of pretreatment data was usually ex post facto and used data sources not designed for research purposes. Collection of posttreatment data was often more systematic but resulted in bias because of poor locating and follow-up practices, and because temporal relationships between admission, length of therapy and time elapsed since treatment were either not reported or not controlled. Several requirements for reporting evaluative research were also discussed; the setting in which a study is carried out, type or types of treatment employed, measurement instruments, and population and sample characteristics should be clearly described. Results should be presented in such a manner that the reader is able to draw his own conclusions from the data as presented. Many of the studies did not pretend to be experimental research. Even as gross evaluations, however, they invariably suffered in one and usually more methodological requirements because of the fact that they had been planned retrospectively rather than prior to treatment. Several sources which may be of assistance in planning such research are given.