Toward ICE-XRF fusion: real-time pose estimation of the intracardiac echo probe in 2D X-ray using deep learning

Purpose Valvular heart disease affects 2.5% of the general population and 10% of people aged over 75, with many patients untreated due to high surgical risks. Transcatheter valve therapies offer a safer, less invasive alternative but rely on ultrasound and X-ray image guidance. The current ultrasound technique for valve interventions, transesophageal echocardiography (TEE), requires general anesthesia and has poor visibility of the right side of the heart. Intracardiac echocardiography (ICE) provides improved 3D imaging without the need for general anesthesia but faces challenges in adoption due to device handling and operator training. Methods To facilitate the use of ICE in the clinic, the fusion of ultrasound and X-ray is proposed. This study introduces a two-stage detection algorithm using deep learning to support ICE-XRF fusion. Initially, the ICE probe is coarsely detected using an object detection network. This is followed by 5-degree-of-freedom (DoF) pose estimation of the ICE probe using a regression network. Results Model validation using synthetic data and seven clinical cases showed that the framework provides accurate probe detection and 5-DoF pose estimation. For the object detection, an F1 score of 1.00 was achieved on synthetic data and high precision (0.97) and recall (0.83) for clinical cases. For the 5-DoF pose estimation, median position errors were found under 0.5mm and median rotation errors below 7.2∘ 7 . 2 ∘ . Conclusion This real-time detection method supports image fusion of ICE and XRF during clinical procedures and facilitates the use of ICE in valve therapy.