How should we report outcomes in reproductive medicine?

Infertility, defined as the inability to conceive within 1 year of unprotected intercourse1, is estimated to affect as many as 186 million people worldwide2. Reproductive medicine aims to help these couples by offering them the best treatment option. As individual perception of the effectiveness of any treatment is affected by bias, it is generally accepted that clinical practice should be based on scientific methods. The definition of evidence-based medicine was first introduced by Sackett as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients3-5. To achieve this, knowledge should be sought in the medical literature to inform clinical decisions6. Reporting the most relevant and informative outcomes in clinical research, therefore, is mandatory for effective comparison between different interventions6. In 2003, the European Society for Human Reproduction and Embryology recommended that the outcome measure of assisted reproductive techniques (ART) and non-ART should be ‘singleton live birth’7. Nothing, however, was stated with respect to the treatment cycle or effectiveness over time. Traditionally, success rates of in-vitro fertilization (IVF) have been reported in terms of live births per fresh cycle or embryo transfer. However, with the increasing use of embryo freezing and thawing, it is essential that outcomes are reported not only following fresh but also after frozen embryo transfer as a complete measure of success of IVF treatment8. Moreover, evaluations should be extended over treatment strategies that incorporate multiple treatment cycles, even if they are of different types, and should also consider any natural conceptions that occur in between treatments. Here, we advocate that, in both clinical trials and clinical practice, the outcome of fertility treatment should be reported as a function of time and as the outcome of multiple transfers from one cycle or the outcome of multiple cycles in a predefined period. Scholten et al.9 showed recently that only 19% of the published randomized controlled trials (RCTs) on effectiveness of medically assisted reproduction (MAR) with pregnancy as an outcome reported on multiple cycles. Studies on intrauterine insemination and ovulation induction reported on multiple cycles more often (57% and 65% of studies, respectively) than did trials on IVF (7%). IVF trials reported more often on multiple cycles when they were carried out in a multicenter setting (odds ratio (OR), 3.7; 95% CI, 1.1–12.5) or used live birth as the primary outcome (OR, 8.7; 95% CI, 1.8–40.3). Trials on IVF comparing protocol variations reported less often on multiple cycles. Since most RCTs on MAR, especially those on IVF, do not report cumulative pregnancy rates, and since only a minority of women become pregnant in their first cycle, the clinical significance of these trials is limited. Data that fertility clinics report to national registries present a similar picture. Wilkinson et al.10 reviewed recently how treatment success is presented by the Society for Assisted Reproductive Technology and the Human Fertilization and Embryology Authority, finding that live births are reported per cycle, per embryo and per new patient (Table 1). Per cycle involves counting all live-birth events (numerator) per oocyte retrieval cycle (denominator) for each woman in the reporting year. Per embryo transfer involves counting all live-birth events (numerator) per embryo transfer (denominator) in each woman in the reporting year. Per new patient involves counting all live births (numerator) per new patient entering the clinic (denominator) in the reporting year. The predominant means of reporting in different national schemes is success per cycle10, 11. Reporting of success rates of the stimulated cycle only is disadvantageous for strategies in which transfer is delayed after freezing, whether freezing be to preserve all embryos or to facilitate single embryo transfer. Heijnen et al.12 suggested, more than a decade ago, a move away from considering outcome in terms of single IVF cycles, towards assessing treatment strategies as a whole over a longer period of time. They proposed that the measure of effectiveness should be the number of couples that conceive a pregnancy resulting in live birth of at least one baby in a specified period after starting treatment, an idea that has been echoed recently8. In presenting the results of a treatment, one could choose the conception rate leading to live birth, i.e. count the moment of conception, and disregard those conceptions that did not lead to live birth. This avoids a follow-up period that starts with 9 months without events, as women not pregnant at the start of treatment usually take at least 9 months to deliver a baby. Studies assessing effectiveness in MAR should therefore aim to study the effectiveness of treatment over multiple cycles, as this reflects clinical practice and captures the relevant perspective for the couple. Although we are not aware of studies measuring the preference of infertile couples, it seems intuitive that couples visiting a fertility clinic desire a baby, preferably healthy; how the baby is conceived and how fast this happens is of less importance. Women undergoing IVF prefer to have a disabled child rather than have no pregnancy13. Indeed, the woman hoping to conceive but struggling with infertility might already be a mother in spirit; for her, not conceiving involves loss of the child that will never be and of the prospect of true motherhood. While, for more than a decade, an argument has been made for reporting on the effectiveness of ART strategies by presenting the occurrence of conception leading to live birth over time, the introduction of frozen embryo transfer has unmasked the flaws and problems of reporting per transfer or per cycle. While, originally, embryo transfer was limited to two embryos, with the other embryos being frozen, a decade ago single embryo transfer reduced the multiple-pregnancy rate14. Recently, the hypothesis that transfers in the non-stimulated cycle instead of the fresh cycle would increase the success rates of IVF, has led to a freeze-all policy15-18. While there is as yet no evidence for superiority of a strategy of frozen compared with fresh embryo transfer in terms of cumulative live-birth rate (meta-analysis of four trials, including 1892 women: OR, 1.09; 95% CI, 0.91–1.31)19, in infertile patients with polycystic ovary syndrome, frozen embryo transfer resulted in a higher live-birth rate (rate ratio, 1.17; 95% CI, 1.05–1.31), mainly due to a decrease in the rate of second-trimester fetal loss after frozen transfer20. The widespread introduction of frozen embryo transfer, separating the collection of eggs and transfer of the embryo, has jeopardized studies and registration systems that report only on the first stimulated cycle, as subsequent frozen transfers resulting in the birth of a baby are not accounted for. While potentially this can be overcome by expressing the success of IVF per cycle started, and reporting all of the frozen cycles resulting from it, the real change needed is to express success as live birth per couple starting treatment. This would effectively bypass the denominator problem that arose after the introduction of frozen transfer. Introduction of a live-birth rate per cycle, without looking at the effectiveness of a treatment strategy over time, is, in our opinion, replacing one mistake with another. Analyzing the impact of a strategy over time needs to become standard practice, not only for clinical research in reproductive medicine but also in registration systems and league tables for individual fertility clinics. While such measurement might not be easy to achieve, the true measure of the quality of a clinic is the percentage of couples that it helps towards a healthy baby, relative to their prognostic profile. This would also facilitate evaluation of personalized delay of treatment in couples with a good prognosis for natural conception21, as well as the use of less invasive treatments, such as ovulation induction22 or intrauterine insemination23. Finally, we would argue that live birth, while being a vital outcome, should not necessarily be the primary one. Importantly, while the aim of reproductive medicine is to assist infertile couples in achieving a baby, the aim of a RCT is to inform infertile couples and clinicians on their best treatment choice. This was demonstrated in a recent review on ovarian stimulation, in which the use of recombinant follicle-stimulating hormone (FSH) plus recombinant luteinizing hormone (LH) was associated with an increased live-birth rate compared with using recombinant FSH alone (four RCTs, including 367 women: OR, 2.46; 95% CI, 1.42–4.25)24. However, for the same comparison, there were many more studies reporting on ongoing pregnancy as an endpoint (12 RCTs, including 1367 women), and this comparison showed a much lower benefit from adding LH, which was not statistically significant. Looking at these results, if we choose to define live birth as the best outcome, then it would appear that using LH in addition to FSH is the best option. It is likely that only the studies that found benefit from adding LH followed up the participants until delivery (4/17 studies). Ongoing pregnancy was, however, reported by 12/17 studies, and therefore more likely to reflect reality. Theoretically, if the intervention under study is thought to influence the chances of ongoing pregnancy occurring, but not its course once established, then conclusions based on ongoing pregnancy will be similar to those based on live birth. Since ongoing pregnancy is reported much more often than is live birth, estimates of treatment effect based on ongoing pregnancy usually have much more precision than do estimates based on live birth25. Indeed, we have shown previously that conclusions drawn from the endpoints pregnancy and ongoing pregnancy are on average similar to those drawn from the endpoint live birth, albeit with more statistical power26. The fact that live birth occurs, by definition, 9 months after treatment, makes it disadvantageous in the monitoring of a trial as compared to ongoing pregnancy. Thus, while live birth should always be reported, it is not necessary to make it the primary outcome of each RCT. Time-robust reporting of reproductive outcome in clinical trials and in fertility clinics occurs infrequently. We suggest that both clinical trials and reports of fertility clinics should present treatment success primarily as a cumulative rate of success per couple starting a treatment. Success should preferably be reported as conception leading to live birth and treatment should include all treatment cycles – both fresh and frozen embryo transfers as well as canceled cycles – per couple entering the trial or fertility clinic.