Joint 3D-reconstruction and background separation in multiple views using graph cuts

This paper deals with simultaneous depth map estimation and background separation in a multi-view setting with several fixed calibrated cameras, two problems which have previously been addressed separately. We demonstrate that their strong interdependency can be exploited elegantly by minimizing a discrete energy functional, which evaluates both properties at the same time. Our algorithm is derived from the powerful "multi-camera scene reconstruction via graph cuts" algorithm presented by Kolmogorov and Zabih (2002). Experiments with both real-world as well as synthetic scenes demonstrate that the presented combined approach yields even more correct depth estimates. In particular, the additional information gained by taking background into account increases considerably the algorithm's robustness against noise.