Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis
Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo
Abstract
Most existing pre-trained language representation models (PLMs) are sub-optimal in sentiment analysis tasks, as they capture the sentiment information from word-level while under-considering sentence-level information. In this paper, we propose SentiWSP, a novel Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks.The word level pre-training task detects replaced sentiment words, via a generator-discriminator framework, to enhance the PLM’s knowledge about sentiment words.The sentence level pre-training task further strengthens the discriminator via a contrastive learning framework, with similar sentences as negative samples, to encode sentiments in a sentence.Extensive experimental results show that SentiWSP achieves new state-of-the-art performance on various sentence-level and aspect-level sentiment classification benchmarks. We have made our code and model publicly available at https://github.com/XMUDM/SentiWSP.
BibTeX
@inproceedings{fan-etal-2022-sentiment,
title = "Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis",
author = "Fan, Shuai and
Lin, Chen and
Li, Haonan and
Lin, Zhenghao and
Su, Jinsong and
Zhang, Hang and
Gong, Yeyun and
Guo, JIan and
Duan, Nan",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.emnlp-main.332/",
doi = "10.18653/v1/2022.emnlp-main.332",
pages = "4984--4994"
}