GELINA: UNIFIED SPEECH AND GESTURE SYNTHESIS VIA INTERLEAVED TOKEN PREDICTION
Téo Guichoux, Théodor Lemerle, Shivam Mehta, Laure Soulier, Catherine Pelachaud, Nicolas Obin
Abstract
Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakening synchrony and prosody alignment. We introduce Gelina, a unified framework that jointly synthesizes speech and co-speech gestures from text using interleaved token sequences in a discrete autoregressive backbone, with modality-specific decoders. Gelina supports multi-speaker and multi-style cloning and enables gesture-only synthesis from speech inputs. Subjective and objective evaluations demonstrate competitive speech quality and improved gesture generation over unimodal baselines.
BibTeX
@inproceedings{icassp2026_gelinaunifiedspe,
title = {GELINA: UNIFIED SPEECH AND GESTURE SYNTHESIS VIA INTERLEAVED TOKEN PREDICTION},
author = {Téo Guichoux and Théodor Lemerle and Shivam Mehta and Laure Soulier and Catherine Pelachaud and Nicolas Obin},
booktitle = {ICASSP 2026},
year = {2026}
}