A Hierarchical Reasoning Framework for Complex Question Answering over Knowledge Graph with Reinforcement Learning

Abstract

Knowledge graph question answering (KGQA) aims to answer natural language questions by reasoning across multiple triples in knowledge graphs (KGs). To enhance model interpretability, reinforcement learning based methods are introduced. However, existing methods struggle with effectively reasoning over KGs for complex questions with constraints. Meanwhile, they commonly encounter delayed and sparse rewards due to weak supervision. This paper proposes a Hierarchical Reasoning Network (HRN) to overcome these challenges. The proposed model dismantles the KGQA task into a high-level process for constraint detection and a low-level process for path reasoning. Moreover, to train from weak supervision, we base the proposed framework on hierarchical Reinforcement Learning with intrinsic motivation to mitigate the issue of low-quality rewards. Furthermore, we employ reward shaping strategy and actor-critic algorithm to effectively optimize our model. Extensive experiments conducted on two benchmark datasets clearly demonstrate the effectiveness of the proposed model.