Sonal Sannigrahi

From Tower to Spire: Adding the Speech Modality to a Translation-Specialist LLM

We introduce Spire, a speech-augmented language model (LM) capable of both translating and transcribing speech input from English into 10 other languages as well as translating text input in both language directions. Spire integrates the speech modality into an existing multilingual LM via speech di