Real‐time by‐example texture synthesis and filtering using local statistics exchange

Abstract Real‐time by‐example texture synthesis is used in interactive virtual worlds to generate the appearance of an unbounded surface from an exemplar texture with as few repetitions as possible. Currently, leading real‐time methods rely on a tiling and blending scheme which is known to synthesize well texture patterns with little spatial organization (such as random scratches or noise) or patterns with periodic spatial organization (such as brick walls or regular tilings). However, attempting to synthesize texture patterns composed of distinct, non‐periodic regions with such methods remains a challenge and can lead to visual artifacts. In this paper, we propose a novel texture synthesis method to address this issue. The key of our technique relies on the observation that exchanging the appearance of periodically tiled regions is sufficient to hide repetition artifacts. We therefore present a synthesis scheme that relies on the real‐time exchange of local statistics, including means, covariance matrices, or histograms. Since our target texture is evaluated on the fly, naive filtering schemes relying on precomputation, such as direct MIP‐mapping, do not provide accurate results. Therefore, we propose an adequate real‐time filtering approximation and show that our method produces high‐quality results with little artifacts and a GPU‐friendly implementation.