Multimodal Sentiment Analysis on Unaligned Sequences Via Holographic Embedding

Abstract

Multimodal sentiment analysis is built on fusion of inputs from multiple modalities. However, at the core of existing fusion method is the dot product between a key vector and a query vector and relies on multiple neural network layers to model the high-order correlation. In this paper, we present a method based on holographic reduced representation which is a compressed version of the outer product to model facilitate higher-order fusion across multiple modality. Experiment shows that our proposal performs promisingly on benchmark multimodal sentiment analysis data sets with improved efficiency.