LumenGSLAM: online physically based rendering with Gaussian Splatting for robust endoscopic reconstruction and tracking

PURPOSE: Realistic online 3D reconstruction from endoscopic video is essential for intraoperative inspection and navigation. However, existing approaches often neglect realistic light modeling, rely on offline optimization, or depend on fragile photometric tracking, thereby limiting physically plausible rendering and stable tracking. This study addresses these limitations through an online framework enabling stable tracking and photorealistic endoscopic rendering. METHODS: We propose LumenGSLAM, an online RGB-D Gaussian Splatting framework for highly texturized dense reconstruction of endoscopic scenes. The method leverages dense depth input to enable stable geometry estimation and photorealistic appearance modeling through per-Gaussian physically based rendering (PBR). Surface-aligned Gaussian initialization and per-parameter gradient scaling are introduced to enhance anatomical fidelity and geometric consistency. Robust camera pose estimation is achieved via a Gaussian-coupled feature-based tracking module using SuperPoint/LightGlue and Perspective-n-Point (PnP), ensuring reliable localization under rapid motion and challenging illumination conditions. RESULTS: ), demonstrating robustness even under large inter-frame motions. CONCLUSION: LumenGSLAM establishes a new benchmark for online RGB-D endoscopic reconstruction, achieving photometrically consistent and anatomically accurate mapping through explicit light modeling and geometry-aware Gaussian optimization. Its robustness makes it a promising candidate for intraoperative navigation and future extensions toward dynamic tissue modeling. Project page: https://github.com/FrancescoLeni/LumenGSLAM .