Cross-modal fusion of external magnetic sensing and simulated 2D imaging for 3D guidewire pose estimation

During percutaneous coronary intervention, conventional 2D X-ray imaging lacks depth information, making it difficult for clinicians to determine the 3D position of the guidewire. While some recent approaches incorporate micro-sensors to assist with pose estimation, many rely on implanted electromagnetic sensors, which can introduce additional clinical risks. In the paper, we present a non-invasive alternative by using an external 3-axis electronic magnetometer array. We further propose a Local-Global Magneto-Visual Network framework (LG-MagNet) that fuses magnetic field information with image data to enable precise 3D pose estimation of the guidewire. Specifically, we first perform a shared encoder for cross-modal feature fusion. Then we employ convolutional operations that integrate local and global features. Finally, we utilize a lightweight prediction head for end-to-end depth regression. We constructed experimental equipment and collected a clinical simulation datasets. Results show a root mean square error (RMSE) of (0.797 ± 0.095 mm) for depth prediction along the Z-axis and an overall RMSE of (1.216 ± 0.072) mm for 3D guidewire shape reconstruction. Quantitative analysis indicates that fusing external magnetometer data with 2D imaging improves pose estimation stability, particularly in regions with curvature.