Reinventing 3D echocardiography: could AI-powered 3D reconstruction from 2D echocardiographic views serve as a viable alternative to 3D probes?

Two-dimensional (2D) echocardiography is one of the most commonly used imaging modalities for assessing the size and function of cardiac chambers. However, parameters derived from 2D echocardiography provide only limited insights into the complex three-dimensional (3D) geometry of the ventricles and atria. This limitation prompted researchers in the 1970s to begin experimenting with technologies that enabled the 3D imaging of the heart using echocardiography.1,2 In its early years, 3D echocardiography relied on a 2D probe that was used to acquire multiple 2D echocardiographic images while it was either tracked in space or moved in a pre-programmed pattern.1,2 The acquired 2D images were then melded into a 3D image. Among the proposed methods, the rotational technique gained the greatest popularity. Assuming that the transducer’s axis remains fixed throughout the scan (i.e. neither the patient nor the housing of the transducer moves), this technique used a rotational device to incrementally rotate the 2D probe and capture 2D videos of a complete cardiac cycle at each position, from which a 3D data set was then reconstructed.1 Nevertheless, all these reconstruction-based techniques soon became obsolete with the invention of high-density matrix array transducers and the advent of real-time 3D echocardiography, leading to the transition of 3D echocardiography from a cumbersome research tool to a user-friendly, clinically applicable diagnostic test. Despite the extensive body of evidence demonstrating its diagnostic and prognostic value,3–5 recent surveys have revealed that 3D echocardiography is still underutilized for assessing left and right ventricular (LV and RV) volumes and ejection fractions.6,7 In addition to limited access to 3D probes and dedicated software packages, the most common reasons for this underutilization include the lack of dedicated training, time constraints, and the complexity of post-processing.6,7 Importantly, the acquired 3D recordings are often unsuitable for 3D analysis due to poor image quality, low temporal resolution, or artefacts from multi-beat acquisition, all representing additional barriers to the widespread use of 3D echocardiography for assessing ventricular volumes and ejection fractions.8,9 Thus, there is a clear need for innovative solutions to overcome these hurdles and enable all patients to benefit from this advanced imaging modality. Drawing inspiration from rotational acquisition methods and harnessing the power of state-of-the-art deep learning techniques, Shen et al.10 proposed CardiacField, a novel tool that utilizes implicit neural representations to generate a 3D model of the heart from echocardiographic videos acquired with a 2D probe manually rotated around the apex of the heart. The tool then uniformly samples 2D slices parallel to the apical four-chamber view from these 3D reconstructions and segments both ventricles to calculate their volumes and ejection fractions. The authors found that CardiacField could accurately reconstruct the heart and achieved a higher peak signal-to-noise ratio than PlaneInVol,11 a conventional interpolation method. Additionally, they evaluated the proposed tool’s usability among users with no or limited experience in echocardiography and observed that CardiacField maintained high image quality. Most importantly, CardiacField predicted LV and RV ejection fractions (LVEF and RVEF) with mean absolute errors (MAEs) of 2.48 and 2.65 percentage points, respectively, outperforming two recently published deep learning models: EchoNet-Dynamic12 (MAE for predicting LVEF: 4.45 percentage points) and RVENet13 (MAE for predicting RVEF: 5.20 percentage points). Implicit neural representations encode 3D shapes or scenes as continuous functions using neural networks rather than represent them as discrete data like meshes or point clouds.14,15 These approaches enable high-fidelity reconstructions while requiring minimal storage, making them an ideal choice for the core of CardiacField.14,15 As these representations are continuous, CardiacField allows for slicing the 3D heart model at any pixel position and angle to generate new views.14,15 Additionally, CardiacField offers several key features that make it an appealing alternative to real-time 3D echocardiography, e.g. it eliminates the need for a 3D probe and post-processing software packages, can be easily operated by novice users as it requires minimal training, and works effectively with 2D videos acquired even using handheld ultrasound devices. The authors should be commended for benchmarking CardiacField against the EchoNet-Dynamic12 and RVENet13 models. However, caution is warranted when interpreting the results of these comparisons due to the fundamental differences between the models. The most important difference that should be considered is that CardiacField integrates data from multiple 2D slices, whereas EchoNet-Dynamic and RVENet infer ejection fractions from a single 2D echocardiographic view.12,13 Moreover, EchoNet-Dynamic was trained to predict LVEF assessed using 2D rather than 3D echocardiography,12 and even though RVENet was trained to predict 3D echocardiography-derived RVEF, it uses an entirely segmentation-free approach.13 Considering all these differences, it is unsurprising that CardiacField estimates 3D ejection fractions with lower errors than the other two models. Although CardiacField is a promising tool, there is still room for improvement. First, the authors recommend scanning patients for 5–10 min to acquire sufficient 2D videos for reconstruction, which demands substantial extra effort from the echocardiographer. Thus, given the time constraints of transthoracic echocardiographic examinations, echocardiographers may opt for simpler and quicker solutions (e.g. EchoNet-Dynamic12 or RVENet13) that use 2D echocardiographic videos acquired routinely as part of the already implemented scanning protocol and do not place any additional burden on them. Acknowledging this limitation, the authors plan to experiment with advanced generative artificial intelligence techniques, with the ambitious goal of reducing the required number of 2D views to 10 or fewer. Achieving this goal without increasing errors to a clinically unacceptable level would be a crucial step for CardiacField in becoming a viable alternative to real-time 3D echocardiography. Second, CardiacField has been evaluated only in a limited number of patients, so further testing in additional populations is warranted. It is also important to note that patients with suboptimal echocardiographic windows and poor image quality were excluded. Therefore, further scrutiny is required to determine whether CardiacField could reliably assess 3D volumes and ejection fractions in these technically challenging populations and, hence, be used as a substitute for real-time 3D echocardiography, which is hindered by these factors. Additionally, it should be thoroughly explored how atrial fibrillation or other arrhythmias at the time of scanning affect the reconstruction process and the accuracy of the predictions. Last, since CardiacField was found to underestimate the true RV volumes (most likely due to the omission of the RV outflow tract), it would also be worth investigating whether acquiring an additional set of RV-focused 2D recordings could resolve this issue. In conclusion, Shen et al.10 have successfully revisited the rotational acquisition methods and combined them with advanced deep learning techniques to develop CardiacField, offering a compelling alternative to real-time 3D echocardiography. While CardiacField has shown impressive performance in reconstructing 3D images and predicting LVEF and RVEF, its widespread adoption will hinge on overcoming several key challenges. Even if these are successfully addressed, its use may remain limited in tertiary centres where 3D echocardiography is already available. Nonetheless, there is little doubt that CardiacField can help democratize 3D echocardiography, and we look forward to seeing how it unfolds its full potential and whether it finds its place in the diagnostic imaging armamentarium. Project number RRF-2.3.1-21-2022-00004 (MILAB) has been implemented with support from the European Union. TKP2021-NVA-12 has been implemented with the support provided by the Ministry of Culture and Innovation of Hungary from the National Research, Development, and Innovation Fund, financed under the TKP2021-NVA funding scheme. M.T. is supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences.