Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL
Songyuan Zhang, Oswin So, Mitchell Black, Zachary Serlin, Chuchu Fan
Abstract
Tasks for multi-robot systems often require the robots to collaborate and complete a team goal while maintaining safety. This problem is usually formalized as a Constrained Markov decision process (CMDP), which targets minimizing a global cost and bringing the mean of constraint violation below a user-defined threshold. Inspired by real-world robotic applications, we define safety as zero constraint violation. While many safe multi-agent reinforcement learning (MARL) algorithms have been proposed to solve CMDPs, these algorithms suffer from unstable training in this setting. To tackle this, we use the epigraph form for constrained optimization to improve training stability and prove that the centralized epigraph form problem can be solved in a distributed fashion by each agent. This results in a novel centralized training distributed execution MARL algorithm which we name
BibTeX
@inproceedings{rss2025_solvingmultiagen,
title = {Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARL},
author = {Songyuan Zhang and Oswin So and Mitchell Black and Zachary Serlin and Chuchu Fan},
booktitle = {RSS 2025},
year = {2025}
}