BYOL-Explore: Exploration by Bootstrapped Prediction
Zhaohan Daniel Guo, Shantanu Thakoor, Miruna Pislar, Bernardo Avila Pires, Florent Altché, Corentin Tallec, Alaa Saade, Daniele Calandriello
Abstract
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually complex environments. BYOL-Explore learns the world representation, the world dynamics and the exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually rich 3-D environment. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.
BibTeX
@inproceedings{
guo2022byolexplore,
title={{BYOL}-Explore: Exploration by Bootstrapped Prediction},
author={Zhaohan Daniel Guo and Shantanu Thakoor and Miruna Pislar and Bernardo Avila Pires and Florent Altch{\'e} and Corentin Tallec and Alaa Saade and Daniele Calandriello and Jean-Bastien Grill and Yunhao Tang and Michal Valko and Remi Munos and Mohammad Gheshlaghi Azar and Bilal Piot},
booktitle={Advances in Neural Information Processing Systems},
editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
year={2022},
url={https://openreview.net/forum?id=qHGCH75usg}
}