Clinimetric Dilemmas in Outcome Scales for Mental Disorders

In the mid-1950s, before the era of psychopharmacological drugs, Parloff et al. 1 designed an outcome scale by which all neurotic patients and all forms of psychotherapy could be clinimetrically compared. The items in this Hopkins Symptom Checklist (SCL) were selected to measure both bodily and mental distress, covering the symptoms people presented when seeking help at the outpatient clinic for psychotherapy at the University Hospital in Baltimore. In its original version the SCL contained 41 items (SCL-41). These were published by Bech 2 with permission from Jerry Frank. Over the 2 decades from 1954 to 1973 the SCL was enlarged to cover states of phobia, obsessive-compulsive states, depression, anxiety, and panic. The ultimate version, SCL-90 3, thus covered the psychopathological symptoms of the ordinary mental disorders (anxiety, depression, mania, and schizophrenia) but still had as its basic structure bodily distress or somatization, and emotional distress or interpersonal sensitivity.At the start of the psychopharmacological era in the 1960s, clinician-rated scales were preferred, as these were seen as fulfilling the requirements of the authorities for approval of the new antianxiety, antidepressant, or antipsychotic drugs. In this respect two Hamilton scales were developed - the Hamilton Depression Scale (HAM-D) 4 and the Hamilton Anxiety Scale (HAM-A) 5 - as well as the Brief Psychiatric Rating Scale (BPRS) 6 that covered the outcome measure of patients with ordinary mental disorders. It was expected that the DSM-5 7 would adopt the self-reported SCL-90 and the clinician-reported HAM-D, HAM-A, and BPRS 8, but the editors of the DSM-5 found that the dimensional approach was too complicated, probably due to a lack of clinimetric understanding. Another dimensional instrument was also seriously considered by the editors of the DSM-5, namely the Patient-Reported Outcome Measurement Information System (PROMIS), according to Greenberg 9. The US National Institute of Health lay behind the PROMIS in an attempt to provide access to health outcomes in both somatic disorders and mental disorders, including psychosomatic disorders. PROMIS covers 5 health domains: physical functioning, social functioning, emotional distress, pain, and fatigue. The items are selected from these domain banks. For example, in the domain of emotional distress items measuring anxiety and depression are to be found in a bank from which the researcher can select items ordered according to the severity location of the dimension under measurement. This ordering of severity is based on clinimetric item response theory models 10. However, the editors of the DSM-5 rejected PROMIS due to lack of sufficient evidence 9. Another candidate for health outcome measures of somatic and mental disorders was developed by the International Consortium for Health Outcomes Measurement (ICHOM) 11. The ICHOM domain of emotional distress lists a Standard Set for Depression and Anxiety which includes a battery of scales. However, many of these are diagnostic screening scales rather than clinimetrically valid easy-to-use outcome scales. The Psychosocial Index (PSI) actually contains easy-to-use subscales 12, covering psychosomatic components such as stress, coping with stress, symptoms of anxiety and depression, abnormal illness behavior, and health-related quality of life. It seems clinimetrically relevant to compare the PSI with the banks of items from existing internationally well-accepted scales such as the SCL-90, HAM-D, HAM-A, and BPRS, including the easy-to-use WHO-5 well-being scale and the side effect of treatment scale 13. In this clinimetric comparison we need, however, to take the following dilemmas into account. When constructing the first version of the SCL, the SCL-41, Parloff et al. 1 considered the components of bodily and emotional distress to be so subjective that ‘no one can judge it but the patient'. This statement is also valid for the WHO-5 Quality of Life Scale 14. On the other hand, in measurement-based care, which is ideally based on a collaborative contact between the therapist and his or her patient, the drug-related outcome is typically clinician-reported. Thus, in the STAR*D study of measurement-based care in depressed patients receiving antidepressive medication, Trivedi et al. 15 used the HAM-D as the primary outcome measure. In the PSI an attempt at collaboration between the therapist and his or her patient is established as the clinician makes a global rating based on the self-reported components.In the practical outcome evaluation plan it is essential for both therapist and patient to have easy-to-use scales because in the practical routine each clinic visit has a duration of less than 15 min 8. These brief scales are conventionally derived from longer versions used in clinical research, because it is important to compare the results from the practical work with the results obtained in controlled trials on which the treatment has been approved by the authorities. From the SCL-90 we have brief subscales for interpersonal sensitivity, somatization, anxiety and depression, clinimetrically selected by item response theory models 16. From the HAM-D we have the HAM-D6 and from the HAM-A the HAM-A613. In the PSI valid subscales covering the different domains are indicated.When using brief scales, it is clinimetrically preferable to have Likert scales as continuous item quantifiers. In the WHO-5, frequency over the past 2 weeks is the quantifier and is defined on a Likert scale from 0 (at no time) to 5 (all of the time). On the SCL scales the items are measured on an intensity quantifier in which the Likert scale goes from 0 (not at all) to 4 (extremely), which is also the case for the PSI.In the WHO-5 the items measure positive well-being and are positively worded, whereas the items in the SCL-90 measure negative symptoms and are therefore negatively worded. In many scales, items worded in opposite directions are used, implying that some of the items have to be reversed in the scoring process 13. According to De Vellis 17 the disadvantages of items worded in opposite directions outweigh any benefits clinimetrically.In outcome scales the ability to discriminate between patients and controls is the most important criterion-related validity index. The SCL-90 and the PSI on the one hand and the HAM-D6, HAM-A6, and BPRS6 on the other hand have all been shown to possess an adequate discriminatory validity.In the PSI, the subscale measuring abnormal illness behavior focuses on the ability to predict response to treatment. Thus, Mechanic 18 considers illness behavior to be an important predictor of how patients adhere to medical advice and of the course of illness, including response to treatment. Sirri et al. 19 have, among the patient-related variables within illness behavior, identified neuroticism as significantly associated with low adherence to medical treatment. Within mental disorders the Eysenck Neuroticism Scale has been shown to have predictive validity both in studies with anxiety disorder 20 and in studies with depression 21. The type A behavior considered to be a personality trait associated with medical conditions 22 should be regarded as part of the Eysenck extraversion dimension 23,24. When summing up all items into a single score, the items have to reflect one single dimension of severity on which the items are located, in a rank-ordered structure 13. The original version of the SCL, the SCL-41, was actually designed as a unidimensional scale measuring distress or discomfort. Thus, Parloff et al. 1 stated that the total score of the 41 items would show improvement due to treatment if there was a net decline in the intensity of each individual item. The full SCL-90 version was tested by Derogatis et al. 3 by factor analysis, which identified 9 factors: somatization, obsession-compulsive states, interpersonal sensitivity, depression, anxiety, anger-hostility, phobic anxiety, paranoid ideation, and psychoticism. This factor analytic study performed by Derogatis et al. 3 has been criticized in many studies, that of Vassend and Skrondal 25 being the most important. Whereas somatization and interpersonal sensitivity correspond to the original SCL-41, the factor of obsessive-compulsive states includes many clinically unspecific items. From a clinimetric point of view we need to start with the clinically meaningful subscales and investigate the unidimensional scalability through item response theory models. In a general population study 16 and a clinical study 26 we found the somatization subscale to have borderline acceptability from a clinimetric point of view, while the subscale of interpersonal sensitivity was clearly acceptable. Clinically, we selected the anxiety items to cover Freud's concept of ‘Die gemischte Neurose', covering generalized anxiety, panic, phobia, and obsessive-compulsive states. The resulting 8-item SCL anxiety subscale had a high scalability, as did the 6-item SCL depression subscale corresponding to the HAM-D6. The full HAM-D (HAM-D17) is multidimensional like the SCL-90, but the HAM-D6 is unidimensional 13. The items in the PROMIS have undergone a careful evaluation for unidimensionality. The items in the various domains are considered as item banks from which the user can select his or her scale 27.In the large National Institute of Mental Health outcome measure project, the change during the therapeutic intervention from baseline to endpoint was the conventional improvement measure, either as a 50% reduction of scale score or in terms of effect size statistics 28. However, Rush et al. 29 recommended evaluating the percentage of patients remitted at endpoint with a concomitant reporting of the number needed to treat because this is much more relevant and understandable than differences between groups in terms of changes from baseline to endpoint in overall symptom severity. When calculating remission rates, we need to have valid cutoff scores on the rating scales. When establishing the HAM-D17 cutoff score for remission, correspondence to a clinical global assessment below mild depression is recommended 30. When using self-reported questionnaires such as the SCL-90 subscales or the WHO-5 Well-Being Index, mean scores for the general population studies are employed 14,16. The PSI has via the global observer ratings an acceptable cutoff score to indicate remission, in contrast to the PROMIS or the ICHOM battery of scales.In conclusion, outcome scales for mental disorders ensure that when working with different treatment orientations, both researchers and ordinary clinicians are able to compare and integrate the results 30. The PSI should be considered as such a scale. The other outcome scales referred to here should be considered, not as rivals, but as convergent solutions.