Neural Approaches to Automated Speech Scoring of Monologue and Dialogue Responses
Yao Qian, Patrick L. Lange, Keelan Evanini, Robert A. Pugh, Rutuja Ubale, Matthew Mulholland, Xinhao Wang
Abstract
We present Neural Network (NN) approaches to the automated assessment of non-native spontaneous speech in a monologic task and a simulated dialogic task. Three attention-based Bidirectional Long Short-Term Memory (BLSTM) Recurrent Neural Networks (RNN) are employed to learn three dimensions (i.e., delivery, language use, and content) of scoring rubrics for the spoken responses. The prompts or turn history information are encoded to low-dimensional vectors by either a BLSTM-RNN or an end-to-end memory network (MemN2N) and used as the conditions of the inputs of the NN for rating the subscore of content. The three subscores are fused together to generate a holistic score. The experimental results show that our approaches significantly outperform the conventional approaches to speech scoring and the correlations of automatically predicted scores with the reference human scores are higher than human-human agreement levels for both tasks.
BibTeX
@inproceedings{icassp2019_neuralapproaches,
title = {Neural Approaches to Automated Speech Scoring of Monologue and Dialogue Responses},
author = {Yao Qian and Patrick L. Lange and Keelan Evanini and Robert A. Pugh and Rutuja Ubale and Matthew Mulholland and Xinhao Wang},
booktitle = {ICASSP 2019},
year = {2019}
}