Can deep learning automatically predict fetal heart pregnancy with almost perfect accuracy?

Sir, We read with enthusiasm the recent article by Tran et al. reporting the utilization of ‘Deep learning as a predictive tool for fetal heart pregnancy…’. (Tran et al. 2019) The authors report the assembly and compilation of a large dataset consisting of time lapse video files of a total of 8836 embryos that were collected from eight clinics and labeled by fetal heart pregnancy (FHP) outcome. The reported area under the ROC curve (AUC) was astoundingly high, 0.93, indicative of a nearly perfect classifier. In fact, the ROC curve reports a perfect true positive rate 1 obtained with 0.72 specificity. The ability to predict FHP outcome with nearly perfect accuracy solely based on time-lapse sequences of embryo development lacking any information on endometrial receptivity means that the maternal factor in determining blastocyst implantation is negligible relative to embryo quality. However, the claimed prediction of FHP as specified in the title is not supported by the training process. Only 1773 out of 8142 negative FHP-labeled embryos had actually been transferred to the uterus whereas the majority of these embryos (7063 embryos) had been discarded based on ‘failed or abnormal fertilization, grossly abnormal morphology or aneuploidy from preimplantation genetic testing’. The clinical need is the evaluation of the implantation potential of embryos that are suitable for transfer rather than the discarded embryos that failed fertilization, showed abnormal fertilization or exhibited a grossly abnormal morphology. Moreover, FHP is not excluded by aneuploidy and determining negative FHP outcome with no direct evidence is not justified (Hassold and Hunt 2001). To obtain further insight, we performed a simple exercise where a fictitious set of labeled embryos was considered that recapitulates the composition of the dataset used by Tran et al.: 8% positive FHP transferred blastocysts, 12% negative FHP transferred blastocysts and 80% non-transferred discarded embryos. Assuming that the video files of discarded embryos can be distinguished relatively easily from Day 5 transferred blastocysts, we considered accurate identification of all discarded embryos and only random prediction of FHP of the transferred embryos. Such a classifier is not predicting FHP—it distinguishes the dynamic appearance of discarded embryos from blastocysts. Under these assumptions, we also obtained AUC 0.93. With this result, we do not attempt to determine that the reported algorithm by Tran et al. generates random prediction of transferred embryos. However, concerns are raised as to which outcome is actually predicted and to what extent this prediction relates to embryo implantation potential. As long as the essence of the reported classifier remains unresolved, we alert the misleading title inadequately highlighting prediction of fetal heart pregnancy. Y.K.T., A.B.M. and A.B. declare the ownership of a pending PCT patent application number PCT/IL2020/050120 protecting a machine-learning based assisted reproductive technology. A.B.M. is a medical consultant for Fairtility Ltd.