Combining single view recognition and multiple view stereo for architectural scenes

This paper describes a structure from motion and recognition paradigm for generating 3D models from 2D sets of images. In particular we consider the domain of architectural photographs. A model based approach is adopted with the architectural model built from a "Lego kit" of parameterised parts. The approach taken is different from traditional stereo or shape from X approaches in that identification of the parameterised components (such as windows, doors, buttresses etc) from one image is combined with parallax information in order to generate the 3D model. This model based approach has two main benefits: first, it allows the inference of shape and texture where the evidence from the images is weak; and second, it recovers not only geometry and texture but also an interpretation of the model, which can be used for automatic enhancement techniques such as the application of reflective textures to windows.