Clinical studies in surgical journals—have we improved?

A critical appraisal of all clinical studies published in 1980 and 1990 in three surgical journals--Diseases of the Colon and Rectum (DCR), Surgery (SURG), and the British Journal of Surgery (BJS)--was made to ascertain the frequency with which various research designs appeared, the standard of individual clinical studies, and a comparison of changes in the past decade. Clinical studies were classified into case studies or comparative studies. Comparative studies included randomized controlled trials (RCT), nonrandomized controlled trials, retrospective cohorts, and case-control studies. A 10-point index score (range, 0-10) was used to assess the comparative studies. A sample of articles was analyzed for interobserver and intraobserver variation, with strong agreement between reviewers for classification of trials (unweighted kappa, 0.87) and index scores (0.67). Of 1,481 articles reviewed, 1,060 were classified as clinical studies. Sixteen percent of all clinical studies were comparative studies in 1980, compared with 17 percent in 1990. Of these, 7 percent were RCT in both years. In 1980, 6 percent of clinical studies in DCR were comparative studies, 19 percent in BJS, and 18 percent in SURG. In 1990, 11 percent, 18 percent, and 18 percent, respectively, were comparative studies. In 1980, the proportion of RCT in DCR was 0 percent, in BJS 12 percent, and in SURG 4 percent, compared with 3 percent, 8 percent, and 8 percent, respectively, in 1990. Overall, 52 of 76 (68 percent) RCT were published in BJS. The standard of comparative studies increased overall from 5.49 to 6.04 (P = NS), and that of RCT increased from 7.06 to 7.70 (P = NS). The standard of comparative studies in DCR in 1980 was lower than those in BJS (P < 0.001) and SURG (P < 0.001). The standard of comparative studies in DCR improved from 1.67 in 1980 to 5.47 in 1990 (P < 0.001). There was no significant difference in the standard of comparative studies among the three journals in 1990. In conclusion, there has been no overall increase in the proportion of stronger clinical trial designs in the journals reviewed. A small increase seen in the overall standard of comparative studies was not statistically significant.