July 28, 2014PLoS ONE124 citationsOpen Access
Accumulating Research: A Systematic Account of How Cumulative Meta-Analyses Would Have Provided Knowledge, Improved Health, Reduced Harm and Saved Resources
Key Result
A systematic review of 50 reports including over 1,500 cumulative meta-analyses demonstrated that assessing existing evidence systematically could have identified treatment effects earlier and prevented unnecessary trials.
Study Design
Type
Systematic Review (n=50)
Structured PICO
P
Population
50 eligible reports including more than 1,500 cumulative meta-analyses of studies of the effects of clinical interventions published from 1992 to 2012
I
Intervention
Cumulative meta-analysis
O
Outcome
Themes illustrating how cumulative meta-analyses could have informed research and reduced waste
Cumulative meta-analyses highlight how reviewing existing evidence can help researchers and funders make more informed decisions, leading to earlier uptake of effective interventions and reduced research waste.
Limitations
- Systematic reviews of healthcare interventions are subject to the impact of selective reporting by researchers.
- There is no widely available system to register prospectively methodological research, making it impossible to determine the extent of selective reporting of cumulative meta-analyses.
Abstract
BACKGROUND: "Cumulative meta-analysis" describes a statistical procedure to calculate, retrospectively, summary estimates from the results of similar trials every time the results of a further trial in the series had become available. In the early 1990 s, comparisons of cumulative meta-analyses of treatments for myocardial infarction with advice promulgated through medical textbooks showed that research had continued long after robust estimates of treatment effects had accumulated, and that medical textbooks had overlooked strong, existing evidence from trials. Cumulative meta-analyses have subsequently been used to assess what could have been known had new studies been informed by systematic reviews of relevant existing evidence and how waste might have been reduced. METHODS AND FINDINGS: We used a systematic approach to identify and summarise the findings of cumulative meta-analyses of studies of the effects of clinical interventions, published from 1992 to 2012. Searches were done of PubMed, MEDLINE, EMBASE, the Cochrane Methodology Register and Science Citation Index. A total of 50 eligible reports were identified, including more than 1,500 cumulative meta-analyses. A variety of themes are illustrated with specific examples. The studies showed that initially positive results became null or negative in meta-analyses as more trials were done; that early null or negative results were over-turned; that stable results (beneficial, harmful and neutral) would have been seen had a meta-analysis been done before the new trial; and that additional trials had been much too small to resolve the remaining uncertainties. CONCLUSIONS: This large, unique collection of cumulative meta-analyses highlights how a review of the existing evidence might have helped researchers, practitioners, patients and funders make more informed decisions and choices about new trials over decades of research. This would have led to earlier uptake of effective interventions in practice, less exposure of trial participants to less effective treatments, and reduced waste resulting from unjustified research.