Answers are grounded in this paper and cite their sources.
Since the early days of IVF, great research efforts have been undertaken to improve the viability of in vitro generated embryos and to improve the selection of embryos from a given pool of embryos for preferential transfer to the uterus. With these efforts came the question of what outcome parameter can best objectively quantify the impact of such interventions. A commonly used outcome parameter in IVF studies is the ‘implantation rate’ (IR). The IR is calculated as [IR = ngestational sacs/ntransferred embryos, where ngestational sacs is the number of gestational sacs observed at vaginal ultrasound 3–5 weeks after transfer and ntransferred embryos is the number of transferred embryos]. This calculation can be performed per patient or aggregated per group of patients: very often the latter approach is chosen and methodological havoc is destined to follow. The distortion of data by the misuse of the IR typically becomes apparent in studies in which, systematically, fewer embryos are transferred in one of the study groups. In studies on embryo selection methods (such as preimplantation genetic screening, blastocyst culture or ‘morphokinetics’), the phenomenon of fewer embryos transferred in the study group comes naturally with the nature of the procedure, for example less embryos are classified as transferable after the intervention. In such a scenario, less embryos being transferred combined with calculating the aggregated IR per group of patients necessarily distort the data towards a higher IR with the intervention, as outlined herein in a hypothetical example. In Fig. 1, 2000 patients are randomized 1:1 to either the study or intervention group. In the study group, in addition to the intervention (in our example preimplantation-genetic-screening by comprehensive chromosome screening: PGS-CCS), an elective single embryo transfer (eSET) is performed, while in the control group a double embryo transfer (DET) is performed. In both groups, the same number of patients is receptive as an effect of randomization. However, the pregnancy rate is, necessarily, statistically significantly higher in the DET group owing to a higher number of embryos being transferred per patient (in our example 55 versus 45%). Furthermore, in the DET group, necessarily, a higher multiple pregnancy rate is observed (in our example 24 versus 0%). While the null hypothesis may well be true in this scenario (e.g. PGS-CCS being totally ineffective), the ‘implantation rate’ is statistically significantly higher in the study group. However, the observation of a higher IR can already be explained by the fact alone, that double the number of embryos has been transferred in the control group, also to all the women with zero conception, and that all these embryos went to the denominator of the calculation of the ‘implantation rate’. In our example, pregnancy rate and IR thereby show divergent effect directions, while both are ‘highly statistically significantly (P < 0.00001)’ different between groups. Example of the distortion of findings by using the outcome parameter IR. Example calculation of the pregnancy rate and implantation rate (IR) in a trial on PGS-CCS in which the null hypothesis is true (e.g. the intervention tested is ineffective) while the ‘implantation rate’ is significantly higher in the study group due to the higher number of embryos transferred in the control group. PGS-CCS: preimplantation genetic screening-comprehensive chromosome screening; eSET: elective single embryo transfer; DET: double embryo transfer; PR: pregnancy rate per patient; IR: implantation rate calculated as the number of implanted embryos divided by the number of embryos transferred per group of patients; Pts: patients. The example in Fig. 1 is an exaggeration, but the confusion introduced by the IR to the published literature is a reality. For example, in a number of RCTs comparing eSET with DET, pregnancy rate and IR show divergent effect directions as in our example in Fig. 1 (Martikainen et al., 2001; Lukassen et al., 2005; Moustafa et al., 2008). However, that obviously does not allow the conclusion to be reached that the ‘implantation potential’ of an embryo is getting better if that embryo is transferred alone. Another, more pertinent, practical example of the distortion of data by misuse of the IR is a recently conducted study on PGS-CCS (Forman et al., 2013). This study has entered systematic reviews (Dahdouh et al., 2015a; Lee et al., 2015) and has been classified as ‘class I evidence’ of the effectiveness of PGS-CCS in increasing IR (Scott et al., 2014; Dahdouh et al., 2015b). Since it is clear that transferring less embryos per embryo transfer reduces the chance of pregnancy, the authors (Forman et al., 2013) hypothesized that the embryo selection method PGS-CCS could level out the chance of pregnancy even if a single embryo is transferred (eSET) as compared with a control group of patients who receive a DET without PGS-CCS. In a nutshell, the study reported that ongoing pregnancy rate is similar between groups and the multiple pregnancy rate is less in the eSET group, while ‘….there was a trend toward a higher sustained implantation rate after single euploid blastocyst transfer (63.2% [55/87] versus 51.7% [89/172]; P = 0.08). Among patients >35 years old ….. the sustained IR was higher in the single euploid blastocyst transfer group (58.3% [28/48] versus 39.8% [39/98]; P = 0.03). This improvement in the IR likely relates to the higher probability of the CCS result overriding traditional embryo morphology for selection (Forman et al., 2013). Surprised by a similar pregnancy rate but a higher IR in this study? Not if you go back to Fig. 1. Other methodological problems with calculating IR per group of patients are that all analyses in a randomized study should refer to the unit that has been randomized. Since, typically, embryos or oocytes cannot be randomized in a clinical trial, it is patients who are the correct unit-of-analysis (Daya, 2003; Vail and Gardener, 2003). Otherwise the power of randomization (e.g. the equal distribution of known and unknown confounders for the chance of pregnancy between groups) is lost. Furthermore, in all cases where more than one embryo is transferred at once, it is clear that the likelihood of an embryo to implant is not independent from the likelihood to implant of the sibling embryo which has originated from, and has been transferred to, the same woman (Matorras et al., 2005). Pooling embryos by groups of patients into aggregate numbers creates a pool of non-independent observations, which must not be subjected to those frequently used statistical tests which require independent observations (chi-square, t-test, Mann–Whitney U-test, etc.). In contrast, randomized patients are independent of each other in the likelihood of achieving a pregnancy and are therefore the correct unit-of-analysis (except for rare cases, such as when two women within a study share a partner…). Another problem with the use of IR is the artificial inflation of the sample size in all those studies where—on average—more than one embryo is transferred at once (e.g. 100 patients are randomized to a study group, but 200 embryos are transferred in total). By using aggregated IRs, ‘statistical significance’ is more easily detected, while clinically relevant effects on pregnancy likelihood may be small—a potential source of type I errors (e.g. erroneously concluding solely on statistical grounds that an intervention is effective). However, a quick Medline search reveals numerous examples in the recent literature where all these methodological issues are disregarded. For example, a recent Alpha Scientists in Reproductive Medicine Consensus statement recommends IR (‘….defined as the proportion of fetal hearts relative to the number of embryos transferred’) as a key performance indicator when evaluating IVF laboratory techniques (Alpha Scientists in Reproductive Medicine, 2012). Systematic reviews published in prominent journals, such as BJOG, Fertility and Sterility or RBMonline, potentiate the problem by meta-analyzing incorrectly calculated IRs from multiple studies. To the best knowledge of the author, the Cochrane library is the only publisher to date that systematically disregards IR as an outcome in their analyses. If more than one embryo is transferred at once and the IR is to be used as an outcome, further methodological issues arise, even if the IR is calculated per patient and not aggregated per group of patients. In the scenario where maximally two or maximally three embryos are transferred at once, the event of implantation can only have three (0, 50 and 100%) and four (0, 33, 66 and 100%) states, respectively. As such, IR is an ordinally scaled categorical event. By analyzing the distribution of patients with 0, 50 and 100% implantation (in case, for example, two embryos are transferred), the event of implantation would relate to the appropriate unit-of-analysis that has been randomized, namely the patient, and could be analyzed as an independent event per intention-to-treat (including those patients without embryo transfer). Furthermore, IR would refer to ‘no pregnancy’, ‘singleton pregnancy’ or ‘twin pregnancy’ in a DET trial. However, this approach makes sense only when all patients within a group and between groups receive the same number of embryos—an unrealistic scenario in a clinical setting except for a SET trial. Still, some researchers have decided to calculate an arithmetic or geometric mean IR across patients within a group. However, the IR observed across patients having on average two or three embryos transferred will obviously not be normally distributed and a parametric statistical test may not be used. The use of a non-parametric test, such as Mann–Whitney U, is still inappropriate (truly continuous data are required) and may be associated with a loss of power. In efficacy trials minimal variation in patient management, other than the intervention to be tested, is the standard. Accordingly, a SET in all patients would be ideal for that purpose. The IR obviously equates the pregnancy rate under these circumstances making the use of IR redundant in such trials. Clinicians and researchers should be aware of the methodological pitfalls and interpretational difficulties that come with the outcome parameter ‘implantation rate’. Interpretational difficulties are especially pertinent to studies in which systematically unequal numbers of embryos are transferred between groups to be compared and where IRs are calculated on aggregated numbers. Beyond the methodological issues, from a patient-care perspective, IR is nothing but a surrogate outcome, since women are interested in having babies and not implantations. In summary, the IR is more often a source of confusion than enlightenment when searching for efficacious treatment options for our patients. The IR should therefore be abandoned from clinical trial reports. There are complex statistical techniques such as the Generalized Estimating Equation with which the (non-independent) event of implantation can be analyzed appropriately, but that has not been the scope of this short commentary. G.G. has conceptualized, drafted, and finalized this manuscript. There was no funding of this study. G.G. has received consultant/honorarium fees from MSD, Ferring, Glycotope, Serono, Finox, Vitrolife, and IBSA and has served on speaker bureaus for MSD, Ferring, Serono, Vitrolife, and IBSA.
No takes yet. Share an insight, caveat, or question.
Georg Griesinger (2016) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: