Depth Map Super-Resolution via Joint Local Gradient and Nonlocal Structural Regularizations

Depth maps have been widely used in many real world applications, such as human-computer interaction and virtual reality. However, due to the limitation of current depth sensing technology, the captured depth maps usually suffer from low resolution and insufficient quality. In this paper, we propose a depth map super-resolution method via joint local gradient and nonlocal structural regularizations. Depth maps contain mainly smooth areas separated by textures which demonstrate distinct geometry direction characteristic. Motivated by this, we classify depth map patches according to their geometrical directions and learn a compact online dictionary in each class. We further introduce two regularization terms into the sparse representation framework. Firstly, a multi-directional total variation model is proposed to characterize the local patterns in the gradient domain. Secondly, a nonlocal autoregressive model is introduced to provide nonlocal constraint to the local structures, which can effectively restore image details and suppress noise. Quantitative and qualitative evaluations compared with state-of-the-art methods demonstrate that the proposed method achieves superior performance for various configurations of magnification factors and datasets.