Comparison of protocols and registry entries to published reports for randomised controlled trials

Publication of complete information about a trial, including its results, is essential if people are to make well-informed decisions about health care. If the results of a study influence what gets reported and what remains unpublished, this will lead to bias and potentially misleading conclusions by users of the research. Selective reporting of randomised trials is a common problem. For example, it has been shown that important changes can occur in the primary and secondary outcomes between the protocol for a trial and its eventual, published report. In our Cochrane methodology review, we set out to tackle the uncertainty as to whether there is also a problem regarding other aspects of a trial (1). When we began work on the review, we thought that there might be a handful of studies that had used cohorts of randomised trials to compare the content of trial reports with the information contained in their protocols, or entries in a trial registry. We actually found 16 eligible studies that could be included in the review, a further eight studies that are awaiting assessment and four ongoing pieces of research. Twelve of the 16 included studies compared protocols to published reports and four based the comparison on trial registry entries. Collectively, they provided a wealth of information on the size of the problem, but because they had created their cohorts in different ways and investigated different aspects of trials, we were not able to do any meta-analyses; but we did detect some key themes running through the studies. Discrepancies between protocols or trial registry entries and published reports were common for eligibility criteria; sample size calculations; allocation concealment; blinding; analyses; subgroup analyses; and methods of handling protocol deviations and missing data; as well as for outcomes. The reasons for the discrepancies were not discussed in the published reports. It is also clear that the problem is widespread. It occurs in trial reports in general medical journals and in high impact factor journals, and with industry and non-industry funded trials. This is not acceptable if the findings of trials are to be used to inform decisions in health care. Full transparency will require protocols to be made publicly available before the results of the trial are known. The quality and extent of information included in trial registries also needs to be improved, including a link to the published protocol. Two ongoing initiatives might help with this. SPIRIT is producing guidelines for the preparation of protocols and COMET is promoting the development of a core outcome set for different clinical areas (http://www.comet-initiative.org). Both of these should lead to better trials, with clearer protocols and less selective reporting, and we look forward to the benefits this will bring to decision makers.