Video mosaics for virtual environments

As computer-based video becomes ubiquitous with the expansion of transmission, storage, and manipulation capabilities, it will offer a rich source of imagery for computer graphics applications. This article looks at one way to use video as a new source of high-resolution, photorealistic imagery for these applications. If you walked through an environment, such as a building interior, and filmed a video sequence of what you saw you could subsequently register and composite the video images together into large mosaics of the scene. In this way, you can achieve an essentially unlimited resolution. Furthermore, since you can acquire the images using any optical technology, you can reconstruct any scene regardless of its range or scale. Video mosaics can be used in many different applications, including the creation of virtual reality environments, computer-game settings, and movie special effects. I present algorithms that align images and composite scenes of increasing complexity-beginning with simple planar scenes and progressing to panoramic scenes and, finally, to scenes with depth variation. I begin with a review of basic imaging equations and conclude with some novel applications of the virtual environments created using the algorithms presented.