WHAT MAKES MATH PROBLEMS HARD FOR REINFORCEMENT LEARNING: A CASE STUDY
Ali Shehper, Anibal M. Medina-Mardones, Lucas Fagan, Bartłomiej Lewandowski, Angus Gruen, Yang Qiu, Piotr Kucharski, Zhenghan Wang
Abstract
Using a long-standing conjecture from combinatorial group theory, we explore, from multiple perspectives, the challenges of finding rare instances carrying disproportionately high rewards. Based on lessons learned in the context defined by the Andrews--Curtis conjecture, we analyze how reinforcement learning agents handle problems of varying hardness. We also address many mathematical questions as a part of our study. Notably, we demonstrate the length reducibility of all but two presentations in the Akbulut--Kirby series (1981), and resolve various potential counterexamples in the Miller--Schupp series (1991), including three infinite subfamilies.
BibTeX
@inproceedings{
shehper2025what,
title={{WHAT} {MAKES} {MATH} {PROBLEMS} {HARD} {FOR} {REINFORCEMENT} {LEARNING}: A {CASE} {STUDY}},
author={Ali Shehper and Anibal M. Medina-Mardones and Lucas Fagan and Bart{\l}omiej Lewandowski and Angus Gruen and Yang Qiu and Piotr Kucharski and Zhenghan Wang and Sergei Gukov},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=KIlw9nWydt}
}