PcLast: Discovering Plannable Continuous Latent States
Anurag Koul, Shivakanth Sujit, Shaoru Chen, Ben Evans, Lili Wu, Byron Xu, Rajan Chari, Riashat Islam
Abstract
Goal-conditioned planning benefits from learned low-dimensional representations of rich observations. While compact latent representations typically learned from variational autoencoders or inverse dynamics enable goal-conditioned decision making, they ignore state reachability, hampering their performance. In this paper, we learn a representation that associates reachable states together for effective planning and goal-conditioned policy learning. We first learn a latent representation with multi-step inverse dynamics (to remove distracting information), and then transform this representation to associate reachable states together in $\ell_2$ space. Our proposals are rigorously tested in various simulation testbeds. Numerical results in reward-based settings show significant improvements in sampling efficiency. Further, in reward-free settings this approach yields layered state abstractions that enable computationally efficient hierarchical planning for reaching ad hoc goals with zero additional samples.
BibTeX
@inproceedings{
koul2024pclast,
title={PcLast: Discovering Plannable Continuous Latent States},
author={Anurag Koul and Shivakanth Sujit and Shaoru Chen and Ben Evans and Lili Wu and Byron Xu and Rajan Chari and Riashat Islam and Raihan Seraj and Yonathan Efroni and Lekan P Molu and Miroslav Dud{\'\i}k and John Langford and Alex Lamb},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=AaTYLZQPyC}
}