A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs

Chen Chu; Yong Li; Jinzhuo Liu; Shuyue Hu; Xuelong Li; Zhen Wang

IJCAI 2022poster36 citations

A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs

Chen Chu, Yong Li, Jinzhuo Liu, Shuyue Hu, Xuelong Li, Zhen Wang

Abstract

Modeling the dynamics of multi-agent learning has long been an important research topic. The focus of previous research has been either on 2-agent settings or well-mixed infinitely large agent populations. In this paper, we consider the scenario where n Q-learning agents locate on regular graphs, such that agents can only interact with their neighbors. We examine the local interactions between individuals and their neighbors, and derive a formal model to capture the Q-value dynamics of the entire population. Through comparisons with agent-based simulations on different types of regular graphs, we show that our model describes the agent learning dynamics in an exact manner.

Agent-based and Multi-agent Systems: Multi-agent LearningAgent-based and Multi-agent Systems: Agent SocietiesAgent-based and Multi-agent Systems: Agent-Based Simulation and Emergence

BibTeX

@inproceedings{ijcai2022p28,
  title     = {A Formal Model for Multiagent Q-Learning Dynamics on Regular Graphs},
  author    = {Chu, Chen and Li, Yong and Liu, Jinzhuo and Hu, Shuyue and Li, Xuelong and Wang, Zhen},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {194--200},
  year      = {2022},
  month     = {7},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2022/28},
  url       = {https://doi.org/10.24963/ijcai.2022/28},
}