An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care

Valid reviews of the effects of mental health care depend on identifying as high a proportion as possible of relevant randomized controlled trials (RCTs). To investigate the sensitivity and precision both of MEDLINE and of hand-searching for RCTs in mental health, 12 journals specializing in mental health and indexed by the National Library of Medicine (NLM) for MEDLINE were searched for the years 1971, 1976, 1981, 1986 and 1991. The sensitivity of the hand-search was 94% (95% Confidence Interval (CI) 93-95%), but it had a precision of only 7% (CI 6-8%). The optimal MEDLINE search had a sensitivity of only 52% (CI 48-56%) and a precision of 59% (CI 55-63%). Of the reports of RCTs identified by the hand-search, 9% (CI 7-11%) were not included in MEDLINE at all. Authors had included methodological descriptions in 84% (CI 80-88%) of RCTs found by the hand-search but missed by the MEDLINE search. Systematic reviews of mental health care which are based solely on MEDLINE searches of the literature will miss a large proportion of the relevant RCTs, and are thus liable to random error and bias. A register of mental health RCTs is urgently required.