SSDM: Scalable Speech Dysfluency Modeling
Jiachen Lian, Xuanru Zhou, Zoe Ezzes, Jet M.J. Vonk, Brittany T. Morin, David Paul Galang Baquirin, Zachary A. Miller, Maria Luisa Gorno-Tempini
Abstract
Speech dysfluency modeling is the core module for spoken language learning, and speech therapy. However, there are three challenges. First, current state-of-the-art solutions~~\cite{lian2023unconstrained-udm, lian-anumanchipalli-2024-towards-hudm} suffer from poor scalability. Second, there is a lack of a large-scale dysfluency corpus. Third, there is not an effective learning framework. In this paper, we propose \textit{SSDM: Scalable Speech Dysfluency Modeling}, which (1) adopts articulatory gestures as scalable forced alignment; (2) introduces connectionist subsequence aligner (CSA) to achieve dysfluency alignment; (3) introduces a large-scale simulated dysfluency corpus called Libri-Dys; and (4) develops an end-to-end system by leveraging the power of large language models (LLMs). We expect SSDM to serve as a standard in the area of dysfluency modeling. Demo is available at \url{https://berkeley-speech-group.github.io/SSDM/}.
BibTeX
@inproceedings{
lian2024ssdm,
title={{SSDM}: Scalable Speech Dysfluency Modeling},
author={Jiachen Lian and Xuanru Zhou and Zoe Ezzes and Jet M.J. Vonk and Brittany T. Morin and David Paul Galang Baquirin and Zachary A. Miller and Maria Luisa Gorno-Tempini and Gopala Anumanchipalli},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=IxEhb4NCvy}
}