Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers

Model pruning has become a useful technique that improves the computational of deep learning, making it possible to deploy solutions in-limited scenarios. A widely-used practice in relevant work assumes a smaller-norm parameter or feature plays a less informative role at the time. In this paper, we propose a channel pruning technique for the computations of deep convolutional neural networks (CNNs) that not critically rely on this assumption. Instead, it focuses on direct of the channel-to-channel computation graph of a CNN without the of performing a computationally difficult and not-always-useful task of high-dimensional tensors of CNN structured sparse. Our approach takes stages: first to adopt an end-to- end stochastic training method that forces the outputs of some channels to be constant, and then to those constant channels from the original neural network by adjusting the of their impacting layers such that the resulting compact model can be fine-tuned. Our approach is mathematically appealing from an perspective and easy to reproduce. We experimented our approach several image learning benchmarks and demonstrate its interesting and competitive performance.