Heavy snow removal in video based on tensor dimensionality reduction and simultaneous sparse coding

We propose a novel snow removal algorithm. First, we make full use of the high-dimensional character- istics of video data and propose a tensor dimensionality reduction-based robust principal component analysis (TDR-RPCA) method to reconstruct clean background, which improves the traditional RPCA's ability to handle four-dimensional data and solves the problems with artifacts residue. Then a penalty regularization term-based outlier detection method is used to detect moving objects and glob- al large snowflakes. A binary mask containing moving object and snowflakes is obtained. According to the mask, we design an adaptive thresholding strategy to remove snowflakes. Our scheme can effectively remove global snowflakes. Considering the sparse characteristic of snowflakes, tinny snowflakes in front of moving objects can be regarded as noise; hence, we deal with these tiny snowflakes using Gaussian-scale mixture-based simultaneous sparse coding. Experiments show that our method can remove snowflakes in videos captured during heavy snow conditions and eliminate artifact residue, which outperforms other state-of-the-art methods.