Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon

How to develop slim and accurate deep neural networks has become crucial for- world applications, especially for those employed in embedded systems. previous work along this research line has shown some promising results, existing methods either fail to significantly compress a well-trained deep or require a heavy retraining process for the pruned deep network to-boost its prediction performance. In this paper, we propose a new layer-wise method for deep neural networks. In our proposed method, parameters of individual layer are pruned independently based on second order of a layer-wise error function with respect to the corresponding. We prove that the final prediction performance drop after pruning bounded by a linear combination of the reconstructed errors caused at each. Therefore, there is a guarantee that one only needs to perform a light process on the pruned network to resume its original prediction. We conduct extensive experiments on benchmark datasets to the effectiveness of our pruning method compared with several-of-the-art baseline methods.