3D Video Billboard Clouds

Abstract 3D video billboard clouds reconstruct and represent a dynamic three‐dimensional scene using displacement‐mapped billboards. They consist of geometric proxy planes augmented with detailed displacement maps and combine the generality of geometry‐based 3D video with the regularization properties of image‐based 3D video. 3D video billboards are an image‐based representation placed in the disparity space of the acquisition cameras and thus provide a regular sampling of the scene with a uniform error model. We propose a general geometry filtering framework which generates time‐coherent models and removes reconstruction and quantization noise as well as calibration errors. This replaces the complex and time‐consuming sub‐pixel matching process in stereo reconstruction with a bilateral filter. Rendering is performed using a GPU‐accelerated algorithm which generates consistent view‐dependent geometry and textures for each individual frame. In addition, we present a semi‐automatic approach for modeling dynamic three‐dimensional scenes with a set of multiple 3D video billboards clouds.