AIConfPaper
← Search

Katrin Tomanek

8 accepted papers

2025

Speech Recognition with LLMs Adapted to Disordered Speech Using Reinforcement Learning

Chirag Nagpal, Subhashini Venugopalan, Jimmy Tobin, Marilyn A. Ladewig, Katherine A. Heller, Katrin Tomanek
ICASSP 2025accepted

We introduce a large language model (LLM) capable of processing speech inputs and show that tuning it further with reinforcement learning on human preference (RLHF) enables it to adapt better to disordered speech than traditional fine-tuning. Our method replaces low-frequency text tokens in an LLM’s…

Cited by 0Source ↗Scholar ↗
2025

Towards a Single ASR Model That Generalizes to Disordered Speech

Jimmy Tobin, Katrin Tomanek, Subhashini Venugopalan
ICASSP 2025accepted

This study investigates the impact of integrating a dataset of disordered speech recordings (~1,000 hours) into the fine-tuning of a near state-of-the-art ASR baseline system. Contrary to what one might expect, despite the data being less than 1% of the training data of the ASR system, we find a con…

Cited by 0Source ↗Scholar ↗
2024

Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics

Tyler A. Chang, Katrin Tomanek, Jessica Hoffmann, Nithum Thain, Erin MacMurray van Liemt, Kathleen Meier-Hellstern +1
COLING 2024main

We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia’s Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives. We frame this as retrieval augmented generation, where perspectives are retrieved…

Cited by 12Source ↗PDF ↗Scholar ↗
2024

Large Language Models As A Proxy For Human Evaluation In Assessing The Comprehensibility Of Disordered Speech Transcription

Katrin Tomanek, Jimmy Tobin, Subhashini Venugopalan, Richard Cave, Katie Seaver, Jordan R. Green +1
ICASSP 2024accepted

Automatic Speech Recognition (ASR) systems, despite significant advances in recent years, still have much room for improvement particularly in the recognition of disordered speech. Even so, erroneous transcripts from ASR models can help people with disordered speech be better understood, especially…

Cited by 0Source ↗Scholar ↗
2023

An Analysis of Degenerating Speech Due to Progressive Dysarthria on ASR Performance

Katrin Tomanek, Katie Seaver, Pan-Pan Jiang, Richard Cave, Lauren Harrell, Jordan R. Green
ICASSP 2023accepted

Although personalized automatic speech recognition (ASR) models have recently been improved to recognize even severely impaired speech, model performance may degrade over time for persons with degenerating speech. The aims of this study were to (1) analyze the change of performance of ASR over time…

Cited by 0Source ↗Scholar ↗
2022

Context-Aware Abbreviation Expansion Using Large Language Models

Shanqing Cai, Subhashini Venugopalan, Katrin Tomanek, Ajit Narayanan, Meredith Morris, Michael Brenner
NAACL 2022long

Motivated by the need for accelerating text entry in augmentative and alternative communication (AAC) for people with severe motor impairments, we propose a paradigm in which phrases are abbreviated aggressively as primarily word-initial letters. Our approach is to expand the abbreviations into full…

Cited by 46Source ↗PDF ↗Scholar ↗
2021

Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech

Katrin Tomanek, Vicky Zayats, Dirk Padfield, Kara Vaillancourt, Fadi Biadsy
EMNLP 2021main

Automatic Speech Recognition (ASR) systems are often optimized to work best for speakers with canonical speech patterns. Unfortunately, these systems perform poorly when tested on atypical speech and heavily accented speech. It has previously been shown that personalization through model fine-tuning…

Cited by 69Source ↗PDF ↗Scholar ↗