A Bayesian treatment of the stereo correspondence problem using half-occluded regions

A half-occluded region in a stereo pair is a set of pixels in one image representing points in space visible to that camera or eye only, and not to the other. These occur typically as parts of the background immediately to the left and right sides of nearby occluding objects, and are present in most natural scenes. Previous approaches to stereo either ignored these unmatchable points or attempted to weed them out in a second pass. An algorithm that incorporates them from the start as a strong clue to depth discontinuities is presented. The authors first derive a measure for goodness of fit and a prior based on a simplified model of objects in space, which leads to an energy functional depending both on the depth as measured from a central cyclopean eye and on the regions of points occluded from the left and right eye perspectives. They minimize this using dynamic programming along epipolar lines followed by annealing in both dimensions. Experiments indicate that this method is very effective even in difficult scenes.>