Qrelation: an Agent Relation-Based Approach for Multi-Agent Reinforcement Learning Value Function Factorization

Siqi Shen; Jun Liu; Mengwei Qiu; Weiquan Liu; Cheng Wang; Yongquan Fu; Qinglin Wang; Peng Qiao

ICASSP 2022accepted0 citations

Qrelation: an Agent Relation-Based Approach for Multi-Agent Reinforcement Learning Value Function Factorization

Siqi Shen, Jun Liu, Mengwei Qiu, Weiquan Liu, Cheng Wang, Yongquan Fu, Qinglin Wang, Peng Qiao

Abstract

The Centralized Training with Decentralized Execution paradigm (CTDE), which trains policies centrally with additional information, is important for Multi-Agent Reinforcement Learning (MARL). For CTDE, value function factorization methods make use of state during training and factorize the value function into multiple local value functions for decentralized execution. These approaches do not fully consider the relational information among agents, resulting in sub-optimal models for complex tasks. To remedy this issue, we propose QRelation which is a graph neural network approach for value function factorization. It considers both the static relations (e.g., agent types) and dynamic relations (e.g., close-by). We show that QRelation can obtain better results than state-of-the-art methods on challenging StarCraft II benchmarks.

BibTeX

@inproceedings{icassp2022_qrelationanagent,
  title = {Qrelation: an Agent Relation-Based Approach for Multi-Agent Reinforcement Learning Value Function Factorization},
  author = {Siqi Shen and Jun Liu and Mengwei Qiu and Weiquan Liu and Cheng Wang and Yongquan Fu and Qinglin Wang and Peng Qiao},
  booktitle = {ICASSP 2022},
  year = {2022}
}