Hadamax Encoding: Elevating Performance in Model-Free Atari
Jacob Eeuwe Kooi, Zhao Yang, Vincent Francois-Lavet
Abstract
Neural network architectures have a large impact in machine learning. However, in the specific case of reinforcement learning, network architectures have remained notably simple, as changes often lead to small gains in performance. This work introduces a novel encoder architecture for pixel-based model-free reinforcement learning. The Hadamax (\textbf{Hada}mard \textbf{max}-pooling) encoder achieves state-of-the-art performance by max-pooling Hadamard products between GELU-activated parallel hidden layers. Based on the recent PQN algorithm, the Hadamax encoder achieves state-of-the-art model-free performance in the Atari-57 benchmark. Specifically, without applying any algorithmic hyperparameter modifications, Hadamax-PQN achieves an 80\% performance gain over vanilla PQN and significantly surpasses Rainbow-DQN. For reproducibility, the full code is available on GitHub.
BibTeX
@inproceedings{
kooi2025hadamax,
title={Hadamax Encoding: Elevating Performance in Model-Free Atari},
author={Jacob Eeuwe Kooi and Zhao Yang and Vincent Francois-Lavet},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=iRQM8Ehgl9}
}