SpaceServe: Spatial Multiplexing of Complementary Encoders and Decoders for Multimodal LLMs
zhicheng li, Shuoming Zhang, Jiacheng Zhao, Siqi Li, Xiyu Shi, Yangyu Zhang, Shuaijiang Li, Donglin Yu
Abstract
Recent multimodal large language models (MLLMs) marry modality-specific vision or audio encoders with a shared text decoder. While the encoder is compute- intensive but memory-light, the decoder is the opposite, yet state-of-the-art serving stacks still time-multiplex these complementary kernels, idling SMs or HBM in turn. We introduce SpaceServe, a serving system that space-multiplexes MLLMs: it decouples all modality encoders from the decoder, and co-locates them on the same GPU using fine-grained SM partitioning available in modern runtimes. A cost-model-guided Space-Inference Scheduler (SIS) dynamically assigns SM slices, while a Time-Windowed Shortest-Remaining-First (TWSRFT) policy batches en- coder requests to minimise completion latency and smooth decoder arrivals. Evaluation shows that SpaceServe reduces time-per-output-token by 4.81× on average and up to 28.9× on Nvidia A100 GPUs. SpaceServe is available at https://github.com/gofreelee/SpaceServe
BibTeX
@inproceedings{
li2025spaceserve,
title={SpaceServe: Spatial Multiplexing of Complementary Encoders and Decoders for Multimodal {LLM}s},
author={zhicheng li and Shuoming Zhang and Jiacheng Zhao and Siqi Li and Xiyu Shi and Yangyu Zhang and Shuaijiang Li and Donglin Yu and Zheming Yang and YUAN WEN and Huimin Cui},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=w4qJ056WhI}
}