IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

In this paper, we are interested in building lightweight and efficient neural networks. Inspired by the success of two design patterns, of structured sparse kernels, e. g. , interleaved group convolutions (IGC), and composition of low-rank kernels, e. g. , bottle-neck modules, we study combination of such two design patterns, using the composition of sparse low-rank kernels, to form a convolutional kernel. Rather than a complementary condition over channels, we introduce a loose condition, which is formulated by imposing the complementary over super-channels, to guide the design for generating a dense kernel. The resulting network is called IGCV3. We empirically that the combination of low-rank and sparse kernels boosts the and the superiority of our proposed approach to the-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and and object detection on COCO.