Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers

Chanyoung Chung, Jaejun Lee, Joyce Jiyoung Whang

TL;DR

A unified framework named HyNT is proposed that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers and significantly outperforms state-of-the-art methods on real-world datasets.

Abstract

A hyper-relational knowledge graph has been recently studied where a triplet is associated with a set of qualifiers; a qualifier is composed of a relation and an entity, providing auxiliary information for a triplet. While existing hyper-relational knowledge graph embedding methods assume that the entities are discrete objects, some information should be represented using numeric values, e.g., (J.R.R., was born in, 1892). Also, a triplet (J.R.R., educated at, Oxford Univ.) can be associated with a qualifier such as (start time, 1911). In this paper, we propose a unified framework named HyNT that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers. We define a context transformer and a prediction transformer to learn the representations based not only on the correlations between a triplet and its qualifiers but also on the numeric information. By learning compact representations of triplets and qualifiers and feeding them into the transformers, we reduce the computation cost of using transformers. Using HyNT, we can predict missing numeric values in addition to missing entities or relations in a hyper-relational knowledge graph. Experimental results show that HyNT significantly outperforms state-of-the-art methods on real-world datasets.

Representation Learning on Hyper-Relational and Numeric Knowledge Graphs with Transformers

TL;DR

A unified framework named HyNT is proposed that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers and significantly outperforms state-of-the-art methods on real-world datasets.

Abstract

A hyper-relational knowledge graph has been recently studied where a triplet is associated with a set of qualifiers; a qualifier is composed of a relation and an entity, providing auxiliary information for a triplet. While existing hyper-relational knowledge graph embedding methods assume that the entities are discrete objects, some information should be represented using numeric values, e.g., (J.R.R., was born in, 1892). Also, a triplet (J.R.R., educated at, Oxford Univ.) can be associated with a qualifier such as (start time, 1911). In this paper, we propose a unified framework named HyNT that learns representations of a hyper-relational knowledge graph containing numeric literals in either triplets or qualifiers. We define a context transformer and a prediction transformer to learn the representations based not only on the correlations between a triplet and its qualifiers but also on the numeric information. By learning compact representations of triplets and qualifiers and feeding them into the transformers, we reduce the computation cost of using transformers. Using HyNT, we can predict missing numeric values in addition to missing entities or relations in a hyper-relational knowledge graph. Experimental results show that HyNT significantly outperforms state-of-the-art methods on real-world datasets.
Paper Structure (34 sections, 9 equations, 5 figures, 13 tables)

This paper contains 34 sections, 9 equations, 5 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Knowledge Graph Embedding with Numeric Literals
  4. N-ary Relations & Hyper-Relational Facts
  5. Transformer-based Knowledge Graph Embedding
  6. Existing Benchmark Datasets
  7. Hyper-Relational Knowledge Graphs Containing Numeric Literals
  8. Definition of a Hyper-Relational and Numeric Knowledge Graph
  9. Creating Real-World Datasets
  10. Link Prediction, Numeric Value Prediction and Relation Prediction Tasks
  11. Learning on Hyper-Relational and Numeric Knowledge Graphs
  12. Representations of Triplets and Qualifiers
  13. Context Transformer
  14. Prediction Transformer
  15. Training of HyNT
  16. ...and 19 more sections

Figures (5)

  • Figure 1: A real-world hyper-relational knowledge graph containing numeric literals. This is a subgraph of HN-WK which is created based on Wikidata. Details are in Section \ref{['sec:rkg']}.
  • Figure 2: Link prediction performance according to the training time on HN-WK. A higher MRR score indicates better performance. HyNT performs better than all the other methods while requiring less training time.
  • Figure 3: Examples of the training procedure of HyNT. We replace a missing component with a mask and train the model to recover the missing component to the ground-truth one.
  • Figure 4: Comparison between HyNT's predictions and the ground-truth values of the total fertility rates of Uruguay and Ghana over time.
  • Figure 5: Comparison between HyNT's predictions and the ground-truth values of rankings of Sweden National Team and India National Team over time.

Theorems & Definitions (1)

  • Definition 1: Hyper-Relational and Numeric Knowledge Graph