Real-Time Projection Mapping Using High-Frame-Rate Structured Light 3D Vision

In this paper, the authors report on the development of a projection-mapping system that can project RGB light patterns that are enhanced for three dimensional (3D) scenes using a graphics processing unit (GPU) based high-frame-rate (HFR) vision system synchronized with HFR projectors. The proposed system can acquire 512 × 512 depth-images in real time at 500 fps. The depth-images processing is accelerated by installing a GPU board for parallel processing of Gray-code structured light illumination using infrared (IR) light patterns projected from an IR projector. Using the computed depth-image, suitable RGB light patterns to be projected are generated in real time for enhanced application tasks. They are projected from an RGB projector as augmented information onto a 3D scene with pixel-wise correspondence even when the 3D scene is time-varied. Experimental results obtained from enhanced application tasks for time-varying 3D scenes such as (1) depth-based color mapping, (2) augmented reality (AR) spirit level and (3) AR wristwatch confirm the efficacy of our system.