MAJoR: Visual Emotion Analysis via Multi-Attribute Joint Reasoning

Abstract

Visual Emotion Analysis (VEA) seeks to anticipate individuals’ emotional reactions to visual stimuli. The subjective perception of visual emotion is an integrated impact of the appearance, scene, and objects presented in an image. It is thus significance to analysis visual emotion by incorporating diverse visual attributes. Motivated by this, in this paper, we propose a novel VEA method based on Multi-Attribute Joint Reasoning (MAJoR). Specifically, we first use a multi-stream networks to learning multi-attribute representations, including the color, brightness, scene type, and object class of an input image. Afterward, we use a Graph Convolution Network (GCN) to model the inherent relationships among such visual attributes, and to predict the emotion category. Finally, we propose a two-stage knowledge distillation strategy, to boost the performance of light-weight VEA models via MAJoR. Extensive experiments conducted on several VEA databases showcase the superiority of the proposed MAJoR model and the distilled lightweight versions, compared to state-of-the-art approaches. Our code and models are available at: https://github.com/AiArt-Gao/MAJoR.