2024
SALM: Speech-Augmented Language Model with in-Context Learning for Speech Recognition and Translation
ICASSP 2024accepted
We present a novel Speech Augmented Language Model (SALM) with multitask and in-context learning capabilities. SALM comprises a frozen text LLM, a audio encoder, a modality adapter module, and LoRA layers to accommodate speech input and associated task instructions. The unified SALM not only achieve…