Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Selective Temporal Knowledge Graph Reasoning

Zhongni Hou, Xiaolong Jin, Zixuan Li, Long Bai, Jiafeng Guo, Xueqi Cheng

TL;DR

An abstention mechanism for TKG reasoning is proposed, which helps the existing models make selective, instead of indiscriminate, predictions, and a confidence estimator is developed to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence.

Abstract

Temporal Knowledge Graph (TKG), which characterizes temporally evolving facts in the form of (subject, relation, object, timestamp), has attracted much attention recently. TKG reasoning aims to predict future facts based on given historical ones. However, existing TKG reasoning models are unable to abstain from predictions they are uncertain, which will inevitably bring risks in real-world applications. Thus, in this paper, we propose an abstention mechanism for TKG reasoning, which helps the existing models make selective, instead of indiscriminate, predictions. Specifically, we develop a confidence estimator, called Confidence Estimator with History (CEHis), to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence. To do so, CEHis takes two kinds of information into consideration, namely, the certainty of the current prediction and the accuracy of historical predictions. Experiments with representative TKG reasoning models on two benchmark datasets demonstrate the effectiveness of the proposed CEHis.

Selective Temporal Knowledge Graph Reasoning

TL;DR

An abstention mechanism for TKG reasoning is proposed, which helps the existing models make selective, instead of indiscriminate, predictions, and a confidence estimator is developed to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence.

Abstract

Temporal Knowledge Graph (TKG), which characterizes temporally evolving facts in the form of (subject, relation, object, timestamp), has attracted much attention recently. TKG reasoning aims to predict future facts based on given historical ones. However, existing TKG reasoning models are unable to abstain from predictions they are uncertain, which will inevitably bring risks in real-world applications. Thus, in this paper, we propose an abstention mechanism for TKG reasoning, which helps the existing models make selective, instead of indiscriminate, predictions. Specifically, we develop a confidence estimator, called Confidence Estimator with History (CEHis), to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence. To do so, CEHis takes two kinds of information into consideration, namely, the certainty of the current prediction and the accuracy of historical predictions. Experiments with representative TKG reasoning models on two benchmark datasets demonstrate the effectiveness of the proposed CEHis.
Paper Structure (22 sections, 14 equations, 4 figures, 3 tables)

This paper contains 22 sections, 14 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. TKG Reasoning Methods
  4. Selective Prediction
  5. Problem Formulation
  6. Formulation of TKG Reasoning
  7. Formulation of Selective TKG Reasoning
  8. Evaluation Metrics of Selective TKG Reasoning
  9. Confidence Estimator with History
  10. The Certainty Scorer
  11. The Historical Accuracy Scorer
  12. The Ranking-based Aggregation
  13. Experiments
  14. Experiment Setting
  15. Experimental Results
  16. ...and 7 more sections

Figures (4)

  • Figure 1: An illustrative diagram of the proposed confidence estimator, CEHis, for selective entity reasoning. For the sake of brevity, the corresponding history $\mathcal{G}_q$ paired with each query $q$ is not explicitly given.
  • Figure 2: Effective reliability results of the selective entity reasoning task on ICEWS14. The penalty $c$ is set to 1, 2, 3, 4 and 5, while the model's tolerance $N$ is set to 5 and 10, respectively.
  • Figure 3: Comparison of variant models of CEHis with different basic TKG reasoning models on ICEWS14.
  • Figure 4: Case study on the necessity of modeling the accuracy of historical predictions. Each number represents how precise $f$'s historical prediction is on the corresponding query.