MARTI: A Framework for Multi-Agent LLM Systems Reinforced Training and Inference
Kaiyan Zhang, Kai Tian, Runze Liu, Sihang Zeng, Xuekai Zhu, Guoli Jia, Yuchen Fan, Xingtai Lv
Abstract
We present MARTI (Multi-Agent Reinforced Training and Inference), an open-source framework designed to facilitate scalable and efficient learning of multi-agent LLM systems. MARTI supports centralized multi-agent interactions and distributed policy training, with the added capability of multi-turn asynchronous rollouts to enhance training efficiency. The framework includes dynamic workflows for multi-agent interactions, which integrate both rule-based verifiable rewards and LLM-based generative rewards. We validate the effectiveness of MARTI through comprehensive experiments on diverse mathematical tasks, demonstrating that multi-agent LLM-based systems outperform single-agent systems within the same inference budget after convergence. Our contributions lay the foundation for exploring scalable collaborations within LLM-based multi-agent systems and advancing the capabilities of large reasoning models.
BibTeX
@inproceedings{
zhang2026marti,
title={{MARTI}: A Framework for Multi-Agent {LLM} Systems Reinforced Training and Inference},
author={Kaiyan Zhang and Kai Tian and Runze Liu and Sihang Zeng and Xuekai Zhu and Guoli Jia and Yuchen Fan and Xingtai Lv and Yuxin Zuo and Che Jiang and Yuru wang and Jianyu Wang and Ermo Hua and Xinwei Long and Junqi Gao and Youbang Sun and Zhiyuan Ma and Ganqu Cui and Ning Ding and Biqing Qi and Bowen Zhou},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=E7jZqo0A50}
}