An improved depth reconstruction method for unstable illumination environments in endoscopic surgeries

In the MIS (minimally invasive surgery), precise measurement and mastery of human organs is very important, even a slight wobble of instrument can cause a great deal of error. Digital 3D reconstruction technology can help doctor to confirm the real distance. Considering depth data is difficult to acquire and annotate on a large scale, especially in the human body scene, so many researches focus on self-supervised network. Most of them are based on a widely adopted hypothesis that Image brightness remains constant in adjacent frames, which cannot be satisfied in the body sense, with unstable lighting conditions in the narrow scale and complicated senses. To solve this problem, this paper proposes a high-precision depth reconstruction method for endoscopic surgical scenarios based on AFNet. Firstly, to overcome the limitations of the constant illumination assumption, we extract local features with illumination invariance and introduce quantization calculations for illumination-invariant feature descriptors in the loss function, to reduce the impact of illumination changes during the supervised process. Secondly, leveraging the synchronous movement of the light source and lens in the endoscope, we establish an association model between the brightness variation and depth prediction. This helps the network grasp the image context and smooth the depth better. Experiments show that our method improves the accuracy of depth estimation in endoscopic environments compared to baseline methods. The reconstruction effects on the ENDOVIS datasets of laparoscopic endoscopy and the ENDOSLAM datasets of gastrointestinal endoscopy are significantly improved.