Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification

Luchen Liu; Sheng Guo; Weilin Huang; Matthew R. Scott

ICASSP 2019accepted0 citations

Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification

Luchen Liu, Sheng Guo, Weilin Huang, Matthew R. Scott

Abstract

Multi-label image classification has achieved remarkable progress thanks to deep convolutional neural networks (CNNs). In this paper, we propose a Decouple Network (DecoupleNet) which is an end-to-end CNN-based framework able to trade off class-level feature independence and relevance during training. The proposed DecoupleNet is able to decouple category-wise independence and relevance with image-level supervision. We design a category-wise space-to-depth module with a spatial pooling strategy to exploit more meaningful convolutional features. They are integrated with class-wise correlated information which is automatically learned via a new self-attention mechanism. We conduct extensive experiments on two large-scale benchmarks: the MS-COCO and the NUS-WIDE, where the proposed DecoupleNet obtains impressive performance compared favorably against the state-of-the-art methods on multi-label image classification.

BibTeX

@inproceedings{icassp2019_decouplingcatego,
  title = {Decoupling Category-wise Independence and Relevance with Self-attention for Multi-label Image Classification},
  author = {Luchen Liu and Sheng Guo and Weilin Huang and Matthew R. Scott},
  booktitle = {ICASSP 2019},
  year = {2019}
}