Decentralized but Not Compromised: Modular Architecture with Refined Observation for Multi-Agent Model-Based Reinforcement Learning

Abstract

Multi-agent adversarial tasks such as swarm robotics and autonomous vehicle coordination, demand efficient decentralized collaboration under partial observability. While model-free multi-agent RL (MF-MARL) methods suffer from necessitating extensive environment interactions, most existing multi-agent model-based RL (MA-MBRL) methods fail to align with the Centralized Training with Decentralized Execution (CTDE) paradigm, which limits system flexibility. This paper proposes a novel modular architecture with refined observations (MARO) to achieve the CTDE paradigm by decoupling agents from the world model. Key innovations include: 1) an enhanced world model with weighted loss and history-augmented rollout for high-quality data generation; 2) a dual-stream semantic decomposition network (DSDN) that performs fine-grained decomposition of observations to refine action mapping and mitigate performance degradation from information loss. Extensive experiments on the StarCraft Multi-Agent Challenge (SMAC) demonstrate superior performance over opponents, validating the effectiveness and advancement of MARO.