Marilyn A. Ladewig

Speech Recognition with LLMs Adapted to Disordered Speech Using Reinforcement Learning

We introduce a large language model (LLM) capable of processing speech inputs and show that tuning it further with reinforcement learning on human preference (RLHF) enables it to adapt better to disordered speech than traditional fine-tuning. Our method replaces low-frequency text tokens in an LLM’s…