Depth-based local registration refinement for augmented reality in pituitary surgery

Purpose Accurate registration between pre-operative anatomical models and intra-operative endoscopic views is essential for reliable Augmented reality (AR) guidance in pituitary surgery. While coarse alignment can be achieved using external facial reconstruction, it does not account for relative motion between the face and internal surgical targets. Precise alignment at the sella region further remains challenging due to the lack of direct depth information in monocular endoscopy. Methods To identify an appropriate similarity metric for registration refinement, we conducted a grid search comparing monocular depth predictions from endoscopic images with depth rendered from a pre-operative 3D model. Normalised cross-correlation (NCC) demonstrated the most robust performance and was selected as the optimisation objective. A continuous optimisation framework was then implemented to iteratively refine translational pose parameters by maximising NCC, with optimisation applied periodically and propagated across frames for computational efficiency. Results Depth-based optimisation improved alignment at the surgical target, increasing DICE from 0.77 to 0.81, IoU from 0.63 to 0.68, and reducing centroid localisation error by 36%. Conclusion Overall, the proposed depth-based refinement improves target alignment in endoscopic AR guidance and demonstrates the potential of monocular depth cues for intra-operative registration correction.