Speech Sentiment Analysis via Pre-Trained Features from End-to-End ASR Models

Zhiyun Lu; Liangliang Cao; Yu Zhang; Chung-Cheng Chiu; James Fan

ICASSP 2020accepted0 citations

Speech Sentiment Analysis via Pre-Trained Features from End-to-End ASR Models

Zhiyun Lu, Liangliang Cao, Yu Zhang, Chung-Cheng Chiu, James Fan

Abstract

In this paper, we propose to use pre-trained features from end-to-end ASR models to solve speech sentiment analysis as a down-stream task. We show that end-to-end ASR features, which integrate both acoustic and text information from speech, achieve promising results. We use RNN with self-attention as the sentiment classifier, which also provides an easy visualization through attention weights to help interpret model predictions. We use well benchmarked IEMOCAP dataset and a new large-scale speech sentiment dataset SWBD-sentiment for evaluation. Our approach improves the-state-of-the-art accuracy on IEMOCAP from 66.6% to 71.7%, and achieves an accuracy of 70.10% on SWBD-sentiment with more than 49,500 utterances.

BibTeX

@inproceedings{icassp2020_speechsentimenta,
  title = {Speech Sentiment Analysis via Pre-Trained Features from End-to-End ASR Models},
  author = {Zhiyun Lu and Liangliang Cao and Yu Zhang and Chung-Cheng Chiu and James Fan},
  booktitle = {ICASSP 2020},
  year = {2020}
}