March 25, 1995Open Access
Misleading meta-analysis
Key Result
Meta-analyses based exclusively on small trials can produce misleading results due to publication bias, as demonstrated by the false-positive meta-analysis of magnesium in acute myocardial infarction.
Structured PICO
Does intravenous magnesium reduce mortality in patients with acute myocardial infarction?
P
Population
Patients with acute myocardial infarction (discussed in the context of historical meta-analyses)
I
Intervention
Intravenous magnesium or streptokinase
C
Comparator
Control/placebo (implied from historical trials)
O
Outcome
Mortalityhard clinical
Meta-analyses based exclusively on small trials can yield highly significant but false-positive results due to publication bias, emphasizing the necessity of large, well-conducted randomized controlled trials.
Limitations
- Publication bias
- Selective non-publication of negative trials
- Inadequate handling of data and analysis
Abstract
A meta-analysis of treatments in myocardial infarction published in 1992 retrospectively showed that streptokinase was associated with a highly significant fall in mortality by 1977, after inclusion of 15 trials.1 Thrombolysis was, however, not widely recommended until 10 years later—after the effect was confirmed in two mega trials.1 2 3 In the case of magnesium, a substantial fall in mortality was evident by 1990, after inclusion of seven trials. In 1993, based on an updated meta-analysis it was argued that magnesium treatment represented an “effective, safe, simple and inexpensive” intervention that should be introduced into clinical practice without further delay.4 The negative results of ISIS 4 (the fourth international study of infarct survival), published in last week's Lancet,5 have dealt a blow to enthusiasm for both magnesium and meta-analysis.6 As the findings of meta-analyses and systematic reviews are generally not tested in mega trials the situation regarding magnesium represents an opportunity to examine a false positive meta-analysis. The table compares the meta-analyses of trials of magnesium and streptokinase after myocardial infarction. Trials were cumulatively included until the treatment effect was significant at P<0.001. For magnesium, seven small trials whose results were published in the 1980s were sufficient to establish the effect. Although trials were larger in the case of fibrinolytic treatment, twice as many studies and two decades were necessary to reach the same level of significance. Until recently it could …