Aspirin for pre‐eclampsia: beware of subgroup meta‐analysis

The aim of systematic reviews of randomized controlled trials (RCTs) is to help achieve consensus about the effects of interventions by summarizing the evidence, increasing the power to detect differential effects, assessing consistency of findings and reducing the risk of bias by using pre-specified, explicit methodology1. When it comes to aspirin therapy for prevention of pre-eclampsia and associated adverse perinatal outcomes, systematic reviews have brought about as much controversy as they have consensus. Research in this area has expanded exponentially over the last three decades, from the first published RCT in 19852. Data are now available on over 37 000 women recruited to more than 55 randomized trials. Researchers have sought to meta-analyze data from aspirin trials on multiple occasions over the years, initially to justify larger trials and then to quantify treatment effects more precisely. Lately, the focus seems to have shifted towards trying to resolve uncertainties through subgroup analyses of available data. Our MEDLINE search for previously published aspirin meta-analyses reporting on either pre-eclampsia or perinatal death identified no fewer than 21 systematic reviews published between 1991 and 2013 (Tables 1 and 2), in addition to one published in this issue of the Journal by Roberge et al.3. The first meta-analysis on aspirin in 1991 summarized data from six small early studies (394 women) and showed that aspirin use in the second and third trimesters significantly reduced the risk of pregnancy-induced hypertension, growth restriction and Cesarean section4. The hypothesis generated was tested in larger trials. Although individual large studies did not show statistically significant benefits with aspirin therapy5-7, subsequent meta-analysis continued to show that aspirin improved pregnancy outcome8. Contradictory findings between large trials and systematic reviews led to the first contentious issue surrounding aspirin: should we believe results of large trials or systematic reviews9, 10? The difficulty in relying on RCTs alone is that they are often not large enough to allow detection of modest effects that may still be clinically meaningful. When trials fail to show significant benefits, insufficient evidence of effect should not be confused with evidence of no effect. For example, CLASP (Collaborative Low-dose Aspirin Study in Pregnancy), the largest aspirin trial to date, which recruited 9356 women, had an incidence of pre-eclampsia of 7.6% in the control group and 6.7% in the aspirin group, with a 12% non-significant reduction in risk of pre-eclampsia5. To have adequate power for this relatively modest effect to reach statistical significance, CLASP would have had to recruit over 25 000 women. Here lies the strength of meta-analysis. Having obtained raw data for 90% of women in aspirin trials (36 trials, 34 288 women), the PARIS Collaboration performed the ‘gold standard’ individual patient data (IPD) meta-analysis11, showing a significant reduction in the risk of pre-eclampsia of 10% (relative risk (RR), 0.90; 95% CI, 0.84–0.97), which is entirely consistent with the risk reduction seen in the CLASP trial. The IPD analysis also showed a significant reduction in the risk of a composite serious adverse outcome (death of mother or baby, small-for-gestational age baby, preterm birth or pre-eclampsia) (RR, 0.90; 95% CI, 0.85–0.96). Importantly, there were no indications of harmful effects on either mother or baby. It was estimated that 67 high-risk women, with a baseline risk of 15%, would need to be treated to prevent one case of serious adverse pregnancy outcome. Although the benefits of aspirin are modest, in the absence of alternative beneficial interventions and with good safety data and low cost, publication of results from the IPD meta-analyses has been followed by greater consensus about recommending aspirin to high-risk women; this is reflected in a number of international guidelines on the management of hypertensive disorders in pregnancy12-16. How consistently these recommendations are being followed in clinical practice globally is difficult to assess, but the use of aspirin is still likely to be patchy. Since the publication of the IPD review11 and that of a large aggregate data Cochrane review including 59 trials (37 560 women)17, a number of systematic reviews with smaller data sets of aspirin trials have been published. One may ask why smaller meta-analyses are continuing to be published, when previously conducted robust systematic reviews have shown with considerable precision the small but consistent benefits of aspirin in pregnancy18. Perhaps we are still trying to find explanations for the large swings in effect size among trials. Maybe we are hoping that if we identify the right group of women to treat, or the right dose or time at which to prescribe it, low-dose aspirin might still be the wonder drug that initial studies suggested it was going to be. It is important to remember that systematic reviews also have limitations and are susceptible to bias. When we are attempting to resolve uncertainties surrounding aspirin through meta-analyses of subgroups, we are dealing with smaller numbers and multiple analyses. This increases susceptibility to bias, and careful interpretation of findings in light of potential limitations becomes crucial to avoid misleading conclusions. It is important to explore subgroup analyses if there are potentially large differences between groups in the risk of a poor outcome with or without treatment, if there is potential heterogeneity of treatment effect in relation to pathophysiology, if there are practical questions about when to treat, or if there are doubts about benefit in specific groups19. However, subgroup analyses are not without hazards, and certain rules need to be followed. Subgroup analyses should be justified carefully and limited to a small number of research questions because there is a chance of finding positive effects purely by chance. As a rule, reports of statistical significance in individual groups should be ignored, because rates of false positives and negatives are high, and the only reliable statistical approach is to perform an interaction test for subgroup-treatment interaction effect. Essentially, subgroup analysis should be seen as ‘hypothesis generating’; the best test for validity of subgroup findings is confirmation in subsequent trials19. It is important to note that the IPD review11 did not find any evidence that any one of their pre-specified subgroups benefited more or less, compared with the others, from the use of aspirin, based either on risk factors or on gestation at randomization before or after 20 weeks, for the outcome of pre-eclampsia. Other meta-analyses evaluating benefits of aspirin in high-risk and low-risk women (risk being based mainly on medical and obstetric history, obstetric factors and ultrasound) have shown results very similar to those in the Cochrane aggregate data review17 for both risk of pre-eclampsia and fetal or perinatal death (Table 1), although in some reviews the risk reduction did not reach statistical significance due to fewer data and more imprecision in the confidence intervals (Table 2). In this issue of the Journal, Roberge et al.3 have amalgamated and updated their previous publications and present an eloquent meta-analysis of selected data comparing the effectiveness of early (≤ 16 weeks) vs late (> 16 weeks) administration of aspirin in reducing the risk of perinatal death and other adverse pregnancy outcomes including pre-eclampsia. Their meta-analysis includes 42 studies, with data on 27 222 women. However, more than 94% of the data are for women in the >16-week group, whilst all 15 trials recruiting women at ≤ 16 weeks were very small (33–350 women randomized; total, 1517). Their results show that aspirin administration at ≤ 16 weeks significantly reduces perinatal death, severe pre-eclampsia and fetal growth restriction. The meta-analysis also shows that aspirin significantly reduces the risk of pre-eclampsia and preterm birth regardless of whether it is given before or after 16 weeks, but benefits appear to be greater if it is given at ≤ 16 weeks. It is important to note that the interaction test between subgroups was statistically significant, suggesting that the differences in the effect size between subgroups were more than would be expected to occur by chance. This is a well-presented meta-analysis that, if the conclusions prove correct, has important implications regarding when we should start prescribing aspirin in pregnancy. However, we feel that the results of this meta-analysis should be interpreted with caution, and cannot be taken as conclusive, because of the inherent limitations of the dataset analyzed. Firstly, as the authors acknowledge, all studies in the ≤ 16-week subgroup were small. The chances of the summary estimate changing significantly with the addition of new data are, therefore, high. Secondly, it is well recognized that meta-analyses of small studies are more likely to overestimate treatment effects due to publication bias: trials with significant findings are more likely to be published, whilst small negative studies remain unpublished and unavailable for inclusion in meta-analysis20. Funnel plots are used to investigate the presence of such publication bias in systematic reviews and, for aspirin meta-analyses, have consistently shown asymmetry, suggesting that small negative trials may be missing21. While it is possible that aspirin may be more beneficial if it is given before 16 weeks, it remains equally possible that, were large trials to be conducted with early aspirin administration, the effects could be far more modest than currently suggested. To illustrate the impact of study size, we have performed a subgroup analysis based on study size for trials that administered aspirin after 16 weeks that were included in the analysis of Roberge et al.3. We used 350 randomized women as the cut-off for study size, as this was comparable to the RCTs included in the ≤ 16-week subgroup. The subgroup analysis clearly demonstrates ‘benefit’ of late aspirin in small RCTs and ‘lack of benefit’ when RCTs are large (Figure 1). The forest plot looks very similar to Figure 2 in the meta-analysis of Roberge et al. Moreover, the meta-analysis by Roberge et al.3 does not include data from all women randomized at ≤ 16 weeks. A number of studies were excluded because gestational age at randomization overlapped 16 weeks. Data on perinatal death from the three largest aspirin trials that recruited women from 12 to 32 weeks5-7 seem to have been available separately for inclusion in the > 16-week subgroup but not in the ≤ 16-week group. A more robust and complete dataset based on gestation at randomization should be available from the PARIS Collaboration IPD data11, and further analyses and publication of those data may help reduce uncertainty in this area. The review could also have been influenced by systematic differences between the populations of women in the two subgroups. There was a marked imbalance between the baseline risk of pre-eclampsia in the control groups for the ≤ 16-week subgroup (17.9%) and the >16-week subgroup (8.4%). For the primary outcome of perinatal death, as the authors acknowledge, definitions were variable and underlying causes unclear. Reports of significant reductions in risk of severe pre-eclampsia with early aspirin administration should also be interpreted with caution, not only in this review but also in other reviews reporting this outcome22. Systematic reviews can overestimate treatment effects when data on important outcomes are missing from a significant number of trials, because statistically significant results have higher odds of being reported than do non-significant findings23, 24. Only a small proportion of eligible trials have reported on severe pre-eclampsia (6/15 in ≤ 16-week and 5/27 in > 16-week groups); therefore, findings are susceptible to reporting bias and conclusions could change significantly if all eligible studies were to report the outcome. Finally, for any subgroup analysis, there should be a clear rationale and justification. Evidence that suggests aspirin may have an effect on placentation is tenuous at best25, 26, and it is equally possible that aspirin may also have a beneficial effect on endothelial dysfunction later in gestation. Even if we assume that aspirin has an effect purely on placentation, it is not clear why 16 weeks has been chosen as the cut-off; the first wave of trophoblast invasion is complete by around 10 weeks, the second wave does not start until 14–15 weeks and active endovascular trophoblast has been seen in uterine myometrial vasculature up to 22 weeks' gestation27, 28. Findings from this most recent review by Roberge et al.3 cannot be taken as conclusive, given the relatively small amount of data, and missing data, in the ≤ 16-week subgroup, as well as the potential impact of other factors such as variable outcome definitions for perinatal death and possible systematic differences between the women in the two gestational age subgroups. It would be reassuring to see these data confirmed by the currently available IPD data. We would appeal to the PARIS Collaboration to make these data publicly available to reduce uncertainty in this area. Although multiple subgroup analyses carry the risk of finding significant effects by chance, we feel the current debate justifies further analysis of the IPD data in this case. Finally, we predict an avalanche of small RCTs and subgroup meta-analyses based on biomarker risk factors as the search to find better tests for prediction of pre-eclampsia continues. Whether women identified in this way benefit from low-dose aspirin will remain unknown for some time. We would, therefore, appeal to those undertaking research in this area to ensure that sample sizes are adequate in order to answer this question with certainty, rather than resorting to subgroup meta-analyses of small trials.