Toward Compositional Generalization in Object-Oriented World Modeling
Linfeng Zhao, Lingzhi Kong, Robin Walters, Lawson L.S. Wong
Abstract
Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.
BibTeX
@InProceedings{pmlr-v162-zhao22b,
title = {Toward Compositional Generalization in Object-Oriented World Modeling},
author = {Zhao, Linfeng and Kong, Lingzhi and Walters, Robin and Wong, Lawson L.S.},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
pages = {26841--26864},
year = {2022},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
volume = {162},
series = {Proceedings of Machine Learning Research},
month = {17--23 Jul},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v162/zhao22b/zhao22b.pdf},
url = {https://proceedings.mlr.press/v162/zhao22b.html},
abstract = {Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.}
}