3 accepted papers
Multimodal foundation models, such as Gemini and ChatGPT, have revolutionized human-machine interactions by seamlessly integrating various forms of data. Developing a universal spoken language model that comprehends a wide range of natural language instructions is critical for bridging communication…
With advancements in large audio-language models (LALMs), which enhance large language models (LLMs) with auditory capabilities, these models are expected to demonstrate universal proficiency across various auditory tasks. While numerous benchmarks have emerged to assess LALMs’ performance, they rem
We introduce a new zero resource code-switched speech bench-mark designed to assess the code-switching capabilities of self-supervised speech encoders directly. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can…