2025
Speech Recognition with LLMs Adapted to Disordered Speech Using Reinforcement Learning
Chirag Nagpal, Subhashini Venugopalan, Jimmy Tobin, Marilyn A. Ladewig, Katherine A. Heller, Katrin Tomanek
ICASSP 2025accepted
We introduce a large language model (LLM) capable of processing speech inputs and show that tuning it further with reinforcement learning on human preference (RLHF) enables it to adapt better to disordered speech than traditional fine-tuning. Our method replaces low-frequency text tokens in an LLM’s…