ACL 2025long0 citations
Soundwave: Less is More for Speech-Text Alignment in LLMs
Yuhao Zhang, Zhiheng Liu, Fan Bu, Ruiyu Zhang, Benyou Wang, Haizhou Li
Abstract
Existing end-to-end speech large language models (LLMs) usually rely on large-scale annotated data for training, while data-efficient training has not been discussed in depth. We focus on two fundamental problems between speech and text: the representation space gap and sequence length inconsistency. We propose Soundwave, which utilizes an efficient training strategy and a novel architecture to address these issues. Results show that Soundwave outperforms other advanced speech LLMs in speech translation and AIR-Bench speech tasks with only a fraction of the training data. Further analysis shows that Soundwave still retains its intelligence during conversation.
BibTeX
@inproceedings{zhang-etal-2025-soundwave,
title = "Soundwave: Less is More for Speech-Text Alignment in {LLM}s",
author = "Zhang, Yuhao and
Liu, Zhiheng and
Bu, Fan and
Zhang, Ruiyu and
Wang, Benyou and
Li, Haizhou",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.acl-long.917/",
doi = "10.18653/v1/2025.acl-long.917",
pages = "18718--18738",
ISBN = "979-8-89176-251-0"
}