Mean-Field Deep Reinforcement Learning for Multi-Agent Path Finding
Lin Huo, Jianlin Mao, Hongjun San, Ruiqi Li, Zhiwei Xuan
Abstract
Continuous-space multi-agent path finding (MAPF) presents severe challenges for deep reinforcement learning (DRL), as joint state–action spaces grow exponentially and fine-grained inter-agent coordination is required. Applying mean-field approximation can reduce input complexity from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(N^{2}~d)$</tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(Nd)$</tex-math></inline-formula>, but naively averaging neighbors may weaken critical local interactions, limiting planning success. To address this, we propose the Mean-Field Multi-Agent Reinforcement Pathfinding Framework (MAMFRPF), which integrates mean-field interaction compression, a dual-stream local mean-field critic, trust-region policy optimization, and a sparse-reward design. The dual-stream critic compensates for information loss caused by mean-field averaging, improving planning success by over 20 percentage points under partial observability, while training time is reduced by roughly 7%. Trust-region optimization ensures monotonic policy improvement, and the sparse-reward mechanism accelerates long-horizon credit assignment. Experiments on continuous MAPF benchmarks show that MAMFRPF outperforms strong baselines by 15% in success rate, scales with agent density, and generalizes across diverse spatial layouts.
BibTeX
@inproceedings{ral2026_meanfielddeeprei,
title = {Mean-Field Deep Reinforcement Learning for Multi-Agent Path Finding},
author = {Lin Huo and Jianlin Mao and Hongjun San and Ruiqi Li and Zhiwei Xuan},
booktitle = {RA-L 2026},
year = {2026}
}