Leveraging Class Abstraction for Commonsense Reinforcement Learning via Residual Policy Gradient Methods

Niklas Hopner; Ilaria Tiddi; Herke van Hoof

IJCAI 2022poster3 citations

Leveraging Class Abstraction for Commonsense Reinforcement Learning via Residual Policy Gradient Methods

Niklas Hopner, Ilaria Tiddi, Herke van Hoof

Abstract

Enabling reinforcement learning (RL) agents to leverage a knowledge base while learning from experience promises to advance RL in knowledge intensive domains. However, it has proven difficult to leverage knowledge that is not manually tailored to the environment. We propose to use the subclass relationships present in open-source knowledge graphs to abstract away from specific objects. We develop a residual policy gradient method that is able to integrate knowledge across different abstraction levels in the class hierarchy. Our method results in improved sample efficiency and generalisation to unseen objects in commonsense games, but we also investigate failure modes, such as excessive noise in the extracted class knowledge or environments with little class structure.

Machine Learning: Deep Reinforcement LearningKnowledge Representation and Reasoning: Common-Sense ReasoningMachine Learning: Reinforcement Learning

BibTeX

@inproceedings{ijcai2022p423,
  title     = {Leveraging Class Abstraction for Commonsense Reinforcement Learning via Residual Policy Gradient Methods},
  author    = {Hopner, Niklas and Tiddi, Ilaria and van Hoof, Herke},
  booktitle = {Proceedings of the Thirty-First International Joint Conference on
               Artificial Intelligence, {IJCAI-22}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Lud De Raedt},
  pages     = {3050--3056},
  year      = {2022},
  month     = {7},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2022/423},
  url       = {https://doi.org/10.24963/ijcai.2022/423},
}