Mixture of States: Routing Token-Level Dynamics for Multimodal Generation
Haozhe Liu, Ding Liu, Mingchen Zhuge, Zijian Zhou, Tian Xie, Sen He, Yukang Yang, Shuming Liu
Abstract
We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a learnable, token-wise router that creates denoising timestep- and input-dependent interactions between modalities' hidden states, precisely aligning token-level features with the diffusion trajectory. This router sparsely selects the top-k hidden states and is trained with an epsilon-greedy strategy, efficiently selecting contextual features with minimal learnable parameters and negligible computational overhead. We validate our design with text-to-image generation (MoS-Image) and editing (MoS-Editing), which achieve state-of-the-art results. With only 3B to 5B parameters, our models match or surpass counterparts up to 4xlarger. These findings establish MoS as a flexible and compute-efficient paradigm for scaling multimodal diffusion models. The project homepage is available at https://haozheliu-st.github.io/mos-homepage/
BibTeX
@inproceedings{cvpr2026_mixtureofstatesr,
title = {Mixture of States: Routing Token-Level Dynamics for Multimodal Generation},
author = {Haozhe Liu and Ding Liu and Mingchen Zhuge and Zijian Zhou and Tian Xie and Sen He and Yukang Yang and Shuming Liu and Yuren Cong and Jiadong Guo and Hongyu Xu and Ke Xu and Kam-Woh Ng and Juan C. Perez and Juan-Manuel Perez-Rua and Tao Xiang and Wei Liu and Shikun Liu and Jürgen Schmidhuber},
booktitle = {CVPR 2026},
year = {2026}
}