Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement
Michael Chang, Alyssa Li Dayan, Franziska Meier, Thomas L. Griffiths, Sergey Levine, Amy Zhang
Abstract
Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations. Worse, the representations of these entities are unknown and must be inferred from sensory percepts. We present a hierarchical abstraction approach to uncover these underlying entities and achieve combinatorial generalization from unstructured visual inputs. By constructing a factorized transition graph over clusters of entity representations inferred from pixels, we show how to learn a correspondence between intervening on states of entities in the agent's model and acting on objects in the environment. We use this correspondence to develop a method for control that generalizes to different numbers and configurations of objects, which outperforms current offline deep RL methods when evaluated on simulated rearrangement tasks.
BibTeX
@inproceedings{
chang2023hierarchical,
title={Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement},
author={Michael Chang and Alyssa Li Dayan and Franziska Meier and Thomas L. Griffiths and Sergey Levine and Amy Zhang},
booktitle={The Eleventh International Conference on Learning Representations },
year={2023},
url={https://openreview.net/forum?id=fGG6vHp3W9W}
}