Real-time variational stereo reconstruction with applications to large-scale dense SLAM

We propose an algorithm for dense and direct large-scale visual SLAM that runs in real-time on a commodity notebook. A fast variational dense 3D reconstruction algorithm was developed which robustly integrates data terms from multiple images. This mitigates the effect of the aperture problem and is demonstrated on synthetic and real data. An additional property of the variational reconstruction framework is the ability to integrate sparse depth priors (e.g. from RGB-D sensors or LiDAR data) into the early stages of the visual depth reconstruction, leading to an implicit sensor fusion scheme for a variable number of heterogenous depth sensors. Embedded into a keyframe-based SLAM framework, this results in a memory efficient representation of the scene and therefore (in combination with loop-closure detection and pose tracking via direct image alignment) enables us to densely reconstruct large scenes in real-time. Experimental validation on the KITTI dataset shows that our method can recover large-scale and dense reconstructions of entire street scenes in real-time from a driving car.