CALM: JOINT CONTEXTUAL ACOUSTIC-LINGUISTIC MODELING FOR PERSONALIZATION OF MULTI-SPEAKER ASR
Muhammad Shakeel, Chyi-Jiunn Lin
Abstract
We present CALM, a joint Contextual Acoustic-Linguistic Modeling framework for multi-speaker automatic speech recognition (ASR). In personalized AI scenarios, the joint availability of acoustic and linguistic cues naturally motivates the integration of target-speaker conditioning with contextual biasing in overlapping conversations. CALM implements this integration in an end-to-end framework through speaker embedding-driven target-speaker extraction and dynamic vocabulary-based contextual biasing. We evaluate CALM on simulated English (LibriSpeechMix) and Japanese (Corpus of Spontaneous Japanese mixtures, CSJMix). On two-speaker mixtures, CALM reduces biased word error rate (B-WER) from 12.7 to 4.7 on LibriSpeech2Mix and biased character error rate (B-CER) from 16.6 to 8.4 on CSJMix2 (eval3), demonstrating the effectiveness of joint acoustic-linguistic modeling across languages. We additionally report results on the AMI corpus (IHM-mix condition) to validate performance on standardized speech mixtures.
BibTeX
@inproceedings{icassp2026_calmjointcontext,
title = {CALM: JOINT CONTEXTUAL ACOUSTIC-LINGUISTIC MODELING FOR PERSONALIZATION OF MULTI-SPEAKER ASR},
author = {Muhammad Shakeel and Chyi-Jiunn Lin},
booktitle = {ICASSP 2026},
year = {2026}
}