Classification of Illumination Methods for Mixed Reality

Abstract A mixed reality (MR) represents an environment composed both by real and virtual objects. MR applications are used more and more, for instance in surgery, architecture, cultural heritage, entertainment, etc. For some of these applications it is important to merge the real and virtual elements using consistent illumination. This paper proposes a classification of illumination methods for MR applications that aim at generating a merged environment in which illumination and shadows are consistent. Three different illumination methods can be identified: common illumination, relighting and methods based on inverse illumination. In this paper a classification of the illumination methods for MR is given based on their input requirements: the amount of geometry and radiance known of the real environment. This led us to define four categories of methods that vary depending on the type of geometric model used for representing the real scene, and the sdifferent radiance information available for each point of the real scene. Various methods are described within their category. The classification points out that in general the quality of the illumination interactions increases with the amount of input information available. On the other hand, the accessibility of the method decreases since its pre‐processing time increases to gather the extra information. Recent developed techniques managed to compensate unknown data with clever techniques using an iterative algorithm, hardware illumination or recent progress in stereovision. Finally, a review of illumination techniques for MR is given with a discussion on important properties such as the possibility of interactivity or the amount of complexity in the simulated illumination.