Enhancing Brain Tumour Aid Diagnosis with Augmented Reality rendering-based 6 DoF Object Pose Estimation

Accurately estimating the position and orientation of objects in augmented reality (AR) is a crucial task known as 6 Degrees of Freedom (6 DoF) object pose estimation. This capability enables machines to understand their surroundings in three dimensions, enabling them to perform various tasks like object recognition, tracking, interaction, and manipulation. In this paper, we propose an enhanced approach for estimating the pose of objects in AR using 6 DoF, specifically focusing on augmenting and visualising brain tumours in 3D to assist with diagnosis. Recently, there has been significant progress in developing generalisable 6 DoF object pose estimators that demonstrate competitive performance. These estimators can accurately determine the pose of an unseen object without requiring specific training or fine-tuning for each new object being tested. We tackle the challenge of estimating 6 DoF poses using only a single RGB image by leveraging a state-of-the-art object pose estimator called Gen6D. By utilising RGB data images and applying Gen6D to estimate the 6 DoF pose of a phantom head texture-less object, we can achieve augmented visualisations of brain tumours. The adoption of deep learning techniques for 6 DoF pose estimation holds immense potential and practical value in the field of medicine, particularly for medical practice and training purposes.