A comparison of ASR and human errors for transcription of non-native spontaneous speech
Matthew Mulholland, Melissa Lopez, Keelan Evanini, Anastassia Loukina, Yao Qian
Abstract
In this paper, we compare ASR and human transcriptions of non-native speech to investigate to what extent the accuracy and the patterns of errors of a modern ASR system match those of human listeners in the context of automated assessment of L2 English language proficiency. We obtained multiple naïve transcriptions of short fragments of non-native spontaneous speech with different proficiency levels using crowdsourcing and matched these against the output of an ASR system. We compare WER and recall at the fragment level and consider human-ASR agreement at the word level. We find that we are able to attain a commensurate level of transcription quality using ASR, but the patterns of errors between the two groups differ at the word level.
BibTeX
@inproceedings{icassp2016_acomparisonofasr,
title = {A comparison of ASR and human errors for transcription of non-native spontaneous speech},
author = {Matthew Mulholland and Melissa Lopez and Keelan Evanini and Anastassia Loukina and Yao Qian},
booktitle = {ICASSP 2016},
year = {2016}
}