Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks

Filter pruning is one of the most effective ways to accelerate and compress neural networks (CNNs). In this work, we propose a global filter algorithm called Gate Decorator, which transforms a vanilla CNN module multiplying its output by the channel-wise scaling factors, i.e. gate. When scaling factor is set to zero, it is equivalent to removing the filter. We use Taylor expansion to estimate the change in the function caused by setting the scaling factor to zero and use the for the global filter importance ranking. Then we prune the network removing those unimportant filters. After pruning, we merge all the scaling into its original module, so no special operations or structures are. Moreover, we propose an iterative pruning framework called-Tock to improve pruning accuracy. The extensive experiments demonstrate effectiveness of our approaches. For example, we achieve the-of-the-art pruning ratio on ResNet-56 by reducing 70% FLOPs without loss in accuracy. For ResNet-50 on ImageNet, our pruned model with40% FLOPs reduction outperforms the baseline model by 0.31% in top-1 accuracy. datasets are used, including CIFAR-10, CIFAR-100, CUB-200, ImageNet-12 and PASCAL VOC 2011. Code is available at.com/youzhonghui/gate-decorator-pruning