Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning

Wenhan Xiong, Thien Hoang, William Yang Wang

TL;DR

This paper reframes multi-hop knowledge graph reasoning as a reinforcement learning problem, using a policy-based agent that operates in a continuous embedding space to construct relational paths.

Abstract

We study the problem of learning to reason in large scale knowledge graphs (KGs). More specifically, we describe a novel reinforcement learning framework for learning multi-hop relational paths: we use a policy-based agent with continuous states based on knowledge graph embeddings, which reasons in a KG vector space by sampling the most promising relation to extend its path. In contrast to prior work, our approach includes a reward function that takes the accuracy, diversity, and efficiency into consideration. Experimentally, we show that our proposed method outperforms a path-ranking based algorithm and knowledge graph embedding methods on Freebase and Never-Ending Language Learning datasets.

DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning

TL;DR

This paper reframes multi-hop knowledge graph reasoning as a reinforcement learning problem, using a policy-based agent that operates in a continuous embedding space to construct relational paths.

Abstract

We study the problem of learning to reason in large scale knowledge graphs (KGs). More specifically, we describe a novel reinforcement learning framework for learning multi-hop relational paths: we use a policy-based agent with continuous states based on knowledge graph embeddings, which reasons in a KG vector space by sampling the most promising relation to extend its path. In contrast to prior work, our approach includes a reward function that takes the accuracy, diversity, and efficiency into consideration. Experimentally, we show that our proposed method outperforms a path-ranking based algorithm and knowledge graph embedding methods on Freebase and Never-Ending Language Learning datasets.

Paper Structure

This paper contains 14 sections, 7 equations, 3 figures, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Reinforcement Learning for Relation Reasoning
  5. Training Pipeline
  6. Bi-directional Path-constrained Search
  7. Experiments
  8. Dataset and Settings
  9. Baselines and Implementation Details
  10. Results
  11. Quantitative Results
  12. Qualitative Analysis of Reasoning Paths
  13. Effect of Supervised Learning
  14. Conclusion and Future Work

Figures (3)

  • Figure 1: Overview of our RL model. Left: The KG environment $\mathcal{E}$ modeled by a MDP. The dotted arrows (partially) show the existing relation links in the KG and the bold arrows show the reasoning paths found by the RL agent. $^{-1}$ denotes the inverse of an relation. Right: The structure of the policy network agent. At each step, by interacting with the environment, the agent learns to pick a relation link to extend the reasoning paths.
  • Figure 2: The distribution of paths lengths on two datasets
  • Figure 3: The success ratio ($succ_{10}$) during training. Task: athletePlaysForTeam.