NOISE-ROBUST AV-ASR USING VISUAL FEATURES BOTH IN THE WHISPER ENCODER AND DECODER
In audiovisual automatic speech recognition (AV-ASR) systems, information fusion of visual features in a pre-trained ASR has been proven as a promising method to improve noise robustness. In this work, based on the prominent Whisper ASR, first, we propose a simple and effective visual fusion method…