Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification

Multi-label image classification has achieved remarkable progress thanks to deep convolutional neural networks (CNNs). In this paper, we propose a Decouple Network (DecoupleNet) which is an end-to-end CNN-based framework able to trade off class-level feature independence and relevance during training. The proposed DecoupleNet is able to decouple category-wise independence and relevance with image-level supervision. We design a category-wise space-to-depth module with a spatial pooling strategy to exploit more meaningful convolutional features. They are integrated with class-wise correlated information which is automatically learned via a new self-attention mechanism. We conduct extensive experiments on two large-scale benchmarks: the MS-COCO and the NUS-WIDE, where the proposed DecoupleNet obtains impressive performance compared favorably against the state-of-the-art methods on multi-label image classification.