Correlated Policy Optimization in Multi-Agent Subteams
Dingyang Chen, Jianing Ye, Zhenyu Zhang, Xiaolong Kuang, Xinyang Shen, Ozalp Ozer, Chongjie Zhang, Qi Zhang
Abstract
In cooperative multi-agent reinforcement learning, agents often face scalability challenges due to the exponential growth of the joint action and observation spaces. Inspired by the structure of human teams, we explore subteam-based coordination, where agents are partitioned into fully correlated subgroups with limited inter-group interaction. We formalize this structure using Bayesian networks and propose a class of correlated joint policies induced by directed acyclic graphs . Theoretically, we prove that regularized policy gradient ascent converges to near-optimal policies under a decomposability condition of the environment. Empirically, we introduce a heuristic for dynamically constructing context-aware subteams with limited dependency budgets, and demonstrate that our method outperforms standard baselines across multiple benchmark environments.
BibTeX
@inproceedings{
chen2026correlated,
title={Correlated Policy Optimization in Multi-Agent Subteams},
author={Dingyang Chen and Jianing Ye and Zhenyu Zhang and Xiaolong Kuang and Xinyang Shen and Ozalp Ozer and Chongjie Zhang and Qi Zhang},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=Tke3BVwUz6}
}