NR-VINS: A Real-Time NIR-RGB Visual-Inertia Odometry in Challenging Illumination Environments

This is the authors' abstract. We don't add key points for this paper.

Traditional visual-inertial odometry (VIO) methods rely on visible light. In challenging illumination conditions such as low-light, no-light, and varying lighting scenarios, visual images fail to feature tracking and localization become difficult. To this end, this article proposes a dual-modal VIO NR-VINS that utilizes both near-infrared (NIR) and visible light images for localization. The system detects and removes noise from NIR image using binary masks and neighboring template filtering to enhance feature tracking on NIR image. In the front end, a robust feature extraction and matching method based on neural networks is designed and deployed, enabling stable feature association and tracking between visible light and NIR images under different lighting conditions through dynamic dual-thresholding. To ensure stable localization in complex lighting environments, a “Hesitant or Decisive” dual-modal image smooth switching method which based on the brightness information from visible light images is devised. The experimental results demonstrate that the proposed method has achieved more accurate localization accuracy than VINS-MONO and ORB-SLAM3. Especially, under challenging illumination conditions where visible light VIO completely fails, the proposed method still achieves an absolute trajectory error (ATE) of 1.5%.