Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, Li Deng

TL;DR

The paper introduces a unified neural-embedding framework for learning entities and relations in large knowledge bases, unifying NTN and TransE under linear/bilinear scoring and showing a simple bilinear model (DistMult) achieves state-of-the-art link prediction on Freebase subsets. It then demonstrates embedding-based rule extraction (EmbedRule) can mine Horn rules by modeling relation composition, outperforming AMIE on compositional reasoning. The work highlights that relation embeddings capture compositional semantics via matrix multiplication, enabling efficient rule discovery and interpretable relation structure. Together, these results establish a scalable approach for both predictive inference and reasoning in multi-relational KBs, with potential for deeper architectures and tensor-based deep models.

Abstract

We consider learning representations of entities and relations in KBs using the neural-embedding approach. We show that most existing models, including NTN (Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized under a unified learning framework, where entities are low-dimensional vectors learned from a neural network and relations are bilinear and/or linear mapping functions. Under this framework, we compare a variety of embedding models on the link prediction task. We show that a simple bilinear formulation achieves new state-of-the-art results for the task (achieving a top-10 accuracy of 73.2% vs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach that utilizes the learned relation embeddings to mine logical rules such as "BornInCity(a,b) and CityInCountry(b,c) => Nationality(a,c)". We find that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication. More interestingly, we demonstrate that our embedding-based rule extraction approach successfully outperforms a state-of-the-art confidence-based rule mining approach in mining Horn rules that involve compositional reasoning.

Embedding Entities and Relations for Learning and Inference in Knowledge Bases

TL;DR

The paper introduces a unified neural-embedding framework for learning entities and relations in large knowledge bases, unifying NTN and TransE under linear/bilinear scoring and showing a simple bilinear model (DistMult) achieves state-of-the-art link prediction on Freebase subsets. It then demonstrates embedding-based rule extraction (EmbedRule) can mine Horn rules by modeling relation composition, outperforming AMIE on compositional reasoning. The work highlights that relation embeddings capture compositional semantics via matrix multiplication, enabling efficient rule discovery and interpretable relation structure. Together, these results establish a scalable approach for both predictive inference and reasoning in multi-relational KBs, with potential for deeper architectures and tensor-based deep models.

Abstract

We consider learning representations of entities and relations in KBs using the neural-embedding approach. We show that most existing models, including NTN (Socher et al., 2013) and TransE (Bordes et al., 2013b), can be generalized under a unified learning framework, where entities are low-dimensional vectors learned from a neural network and relations are bilinear and/or linear mapping functions. Under this framework, we compare a variety of embedding models on the link prediction task. We show that a simple bilinear formulation achieves new state-of-the-art results for the task (achieving a top-10 accuracy of 73.2% vs. 54.7% by TransE on Freebase). Furthermore, we introduce a novel approach that utilizes the learned relation embeddings to mine logical rules such as "BornInCity(a,b) and CityInCountry(b,c) => Nationality(a,c)". We find that embeddings learned from the bilinear objective are particularly good at capturing relational semantics and that the composition of relations is characterized by matrix multiplication. More interestingly, we demonstrate that our embedding-based rule extraction approach successfully outperforms a state-of-the-art confidence-based rule mining approach in mining Horn rules that involve compositional reasoning.

Paper Structure

This paper contains 18 sections, 15 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multi-Relational Representation Learning
  4. Entity Representations
  5. Relation Representations
  6. Parameter Learning
  7. Inference Task I: Link Prediction
  8. Datasets
  9. Implementation details
  10. Results
  11. Inference Task II: Rule Extraction
  12. Background and Notations
  13. Embedding-based Rule Extraction
  14. Experiments
  15. Results
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Aggregated precision of top length-2 rules extracted by different methods
  • Figure 2: Aggregated precision of top length-3 rules extracted by different methods
  • Figure 3: Relation embeddings ( DistAdd)
  • Figure 4: Relation embeddings ( DistMult)