RIFO: Restoring images with fence occlusions

Many scenes, e.g., zoos, parks, and gardens, are guarded by fences, and people can only take pictures through the fences. It is desirable to remove visually-annoying fence occlusions from images. This paper proposes a novel approach to restore images from fence occlusions (RIFO). The proposed method consists of two steps: fence detection, and disocclusion restoration. In fence detection, the image is first clustered into superpixels, which are fitted into rectangles. We collect fence pixels from superpixels by determining the elongation of their fitted rectangles. A primary shape of the fence is obtained by a color-based classifier learned from sampled pixels. Then, multi-RANSAC and moving least squares (MLS) are used for sketching the fence structure. Complete fence is detected by expanding the fence structure. Disoccluded regions are restorated by a patch-based approach using matrix completion. Experimental results show that our method detects complete fences from images, and the disoccluded regions are faithfully recovered, yielding clean and complete images.