Prompt-augmented Feature with Cross-domain Contrastive Learning for Efficient Multi-domain Sentiment Analysis

Abstract

Pre-trained language models (PrLMs) demonstrate impressive performance on the sentiment analysis task. However, the large number of trainable parameters brings about heavy computational costs, which become more serious in multi-domain scenarios. In this paper, we propose to extract multi-layer features from the PrLM for efficient training since the training process is independent to its large backbone. Meanwhile, compared with the conventional feature extraction, we leverage prompts to induce PrLM for generating sentiment-aware features which lead to significant improvement on the sentiment analysis. In addition, most previous methods adopted a domain alignment paradigm for multi-domain learning, which becomes cumbersome when the number of domains is large. Therefore, we propose a novel prompt-augmented cross-domain contrastive learning for generalizable performance, which clusters samples with the same label under different prompts or domains. Our method is evaluated on two public multi-domain sentiment analysis benchmarks, which significantly outperforms recent state-of-the-art methods. Extensive ablation studies also verify the effectiveness of each proposed component.