Robust feature learning by improved auto-encoder from non-Gaussian noised images

Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Net-works(DBN) and stacks of auto-encoder variants, with impressive results obtained in several areas, mostly on vision and languages datasets. These learning algorithms aim to find good representations for data, which can be used for classification, reconstruction, visualization and so on. Despite the progress, most existing algorithms would be fragile to non-Gaussian noises and outliers due to the criterion of mean square error(MSE) and cross entropy(CE). In this paper, we propose a robust auto-encoder called correntropy-based contractive auto-encoder(C-CAE) to learn robust features from data with non-Gaussian noises and outliers. The maximum correntropy criterion(MCC) is adopted as reconstruction cost function and a well chosen penalty term is added to the reconstruction cost function. By replacing cross entropy with MCC, the proposed method can learn robust features from the data containing non-Gaussian noises and outliers. The penalty term corresponds to the Frobenius norm of the Jacobian matrix of the encoder activations with respect to the input. By adding the penalty term, the antinoise ability of the proposed method is improved. The proposed method is evaluated using the MNIST benchmark dataset. Experimental results show that, compared with the traditional auto-encoders, the proposed method learns robust features, improves classification accuracy and reduces the reconstruction error, which demonstrates that the proposed method is capable of learning robust features on noisy data.