Artificial Intelligence-driven Framework for Augmented Reality Markerless Navigation in Knee Surgery

Conventional orthopaedic navigation systems depend on marker-based tracking, which may introduce additional skin incisions, increase the risk and discomfort for the patient, and entail increased workflow complexity. The guidance is conveyed via 2D monitors, which may distract the surgeon and increase the cognitive burden. This study presents an Artificial Intelligence (AI) - driven surgical navigation framework for knee replacement surgery. The system comprises an Augmented Reality (AR) interface that combines an occlusions-robust deep learning-based markerless bone tracking and registration algorithm with a commercial HoloLens 2 headset calibrated for the user's perspective on both eyes. The feasibility of such a system in navigating a bone drilling task is investigated with an experienced orthopaedic surgeon on three cadaveric knees under realistic operating room conditions. After registering an implant model to computed tomography (CT) scans, the preoperative plans are determined based on the location of the fixation pins. Navigation accuracy is quantified using a highly accurate optical tracking system. The achieved drilling error is 7.88±2.41mm in translation and 7.36±1.77° in orientation. The results demonstrate the viability of integrating AI and AR technology to navigate knee surgery.