A Novel Pruning Algorithm Based on Data-Driven and Knowledge-Driven Global Channel Pruning

This is the authors' abstract. We don't add key points for this paper.

Pruning reduces the number of parameters in CNN models, making them lighter while preserving accuracy. Nevertheless, current pruning algorithms do not fit well with the input data, making it hard to recover the accuracy of the network after pruning. To this end, we propose a Data-driven and Knowledge-driven Global Channel Pruning algorithm (DKGCP), which leverages a data-driven channel self-attention module to align channel importance with the input data, enhancing the performance of pruned models, while incorporating knowledge-driven modules for local reconstruction and global feature alignment to improve generalization. Specifically, 1) During pre-training, the data-driven channel self-attention module uses all input data to determine channel importance aligned with the dataset. 2) During model optimization, the local reconstruction feature knowledge-driven module fits intermediate features between the pruned and original networks, while the global warming feature knowledge-driven module aligns global features and combines the cross-entropy loss to guide the feature learning process of the pruned network. In comparative experiments on VGGNet, GoogLeNet, and ResNet, our method outperformed state-of-the-art approaches. Notably, it achieved 94.55% accuracy on the pruned ResNet-110 with a 50% pruning rate on the CIFAR-10 dataset.