Multi-Projector Dynamic Spatially Augmented Reality on Deformable Surfaces

Current multi-projector spatially augmented reality systems are limited to static and/or rigid surfaces. We will demonstrate the first multi-projector spatially augmented reality system on non-rigid surfaces like deformable fabrics and human body. Using two projectors and two RGB-D cameras, our system reconstructs the changing surface shape in realtime and adapts the projection from both projectors automatically to ensure a seamless, registered projection display. Such systems have applications in various domains such as art, design, entertainment and medicine. In particular, we will demonstrate two applications that showcase the flexibility and scale of the system: projection on a large deformable fabric, and a surgical guidance system for cleft palate surgery.