UNeC: Unsupervised Exploring In Controllable Space

Abstract

In unsupervised reinforcement learning, agents traverse a reward-free environment, aiming for rapid generalisation to subsequent tasks. This strategy offers a compelling resolution to the challenges of sample efficiency. Nevertheless, environments are frequently saturated with excessive information. The omnipresence of uncontrollable states, akin to noise, can impede the efficacy of unsupervised exploration. To address this issue, we introduce the UNeC framework, standing for UNsupervised Exploring in Controllable Space. This approach leverages the intricate dependencies between states and actions, establishing constraints on controllable state representation. Building on this foundation, we employ particle entropy to evaluate states, adeptly directing agent exploration. Empirical evaluations on the unsupervised RL benchmark affirm the superiority of our method, with a particular emphasis on its dominance in noise-affected scenarios.