Exploring the Efficacy of Multi-Agent Reinforcement Learning for Autonomous Cyber Defence: A CAGE Challenge 4 Perspective
Mitchell Kiely, Metin Ahiskali, Etienne Borde, Benjamin Bowman, David Bowman, Dirk van Bruggen, KC Cowan, Prithviraj Dasgupta
Abstract
As cyber threats become increasingly automated and sophisticated, novel solutions must be introduced to improve defence of enterprise networks. Deep Reinforcement Learning (DRL) has demonstrated potential in mitigating these advanced threats. Single DRL Agents have proven utility toward execution of autonomous cyber defence. Despite the success of employing single DRL Agents, this approach presents significant limitations, especially regarding scalability within large enterprise networks. An attractive alternative to the single agent approach is the use of Multi-Agent Reinforcement Learning (MARL). However, developing MARL agents is costly with few options for examining MARL cyber defence techniques against adversarial agents. This paper presents a MARL network security environment, the fourth iteration of the Cyber Autonomy Gym for Experimentation (CAGE) challenges. This challenge was specifically designed to test the efficacy of MARL algorithms in an enterprise network. Our work aims to evaluate the potential of MARL as a robust and scalable solution for autonomous network defence.
BibTeX
@article{Kiely_Ahiskali_Borde_Bowman_Bowman_van Bruggen_Cowan_Dasgupta_Devendorf_Edwards_Fitts_Fugate_Gabrys_Gould_Huang_Jacobs_Kerr_King_Li_Martinez_Moir_Murphy_Naish_Owens_Purchase_Ridley_Taylor_Farmer_Valentine_Zhang_2025, title={Exploring the Efficacy of Multi-Agent Reinforcement Learning for Autonomous Cyber Defence: A CAGE Challenge 4 Perspective}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35158}, DOI={10.1609/aaai.v39i28.35158}, abstractNote={As cyber threats become increasingly automated and sophisticated, novel solutions must be introduced to improve defence of enterprise networks. Deep Reinforcement Learning (DRL) has demonstrated potential in mitigating these advanced threats. Single DRL Agents have proven utility toward execution of autonomous cyber defence. Despite the success of employing single DRL Agents, this approach presents significant limitations, especially regarding scalability within large enterprise networks. An attractive alternative to the single agent approach is the use of Multi-Agent Reinforcement Learning (MARL). However, developing MARL agents is costly with few options for examining MARL cyber defence techniques against adversarial agents. This paper presents a MARL network security environment, the fourth iteration of the Cyber Autonomy Gym for Experimentation (CAGE) challenges. This challenge was specifically designed to test the efficacy of MARL algorithms in an enterprise network. Our work aims to evaluate the potential of MARL as a robust and scalable solution for autonomous network defence.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Kiely, Mitchell and Ahiskali, Metin and Borde, Etienne and Bowman, Benjamin and Bowman, David and van Bruggen, Dirk and Cowan, KC and Dasgupta, Prithviraj and Devendorf, Erich and Edwards, Ben and Fitts, Alex and Fugate, Sunny and Gabrys, Ryan and Gould, Wayne and Huang, H. Howie and Jacobs, Jules and Kerr, Ryan and King, Isaiah J. and Li, Li and Martinez, Luis and Moir, Christopher and Murphy, Craig and Naish, Olivia and Owens, Claire and Purchase, Miranda and Ridley, Ahmad and Taylor, Adrian and Farmer, Sara and Valentine, William John and Zhang, Yiyi}, year={2025}, month={Apr.}, pages={28907-28913} }