Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Counterfactual Reasoning with Knowledge Graph Embeddings

Lena Zellinger, Andreas Stephan, Benjamin Roth

TL;DR

This paper model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules, and develops COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluates it on the authors' benchmark.

Abstract

Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.

Counterfactual Reasoning with Knowledge Graph Embeddings

TL;DR

This paper model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules, and develops COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluates it on the authors' benchmark.

Abstract

Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.
Paper Structure (40 sections, 9 equations, 3 figures, 12 tables, 2 algorithms)

This paper contains 40 sections, 9 equations, 3 figures, 12 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. CFKGR: Task Description
  3. Definition of Counterfactual Graphs
  4. Evaluation
  5. CFKGR: Dataset Creation
  6. Data Generating Process
  7. CFKGR-CoDEx
  8. Human Annotation
  9. Counterfactual Reasoning with Knowledge Graph Embeddings
  10. Experiments
  11. General Setup
  12. Results
  13. Case Study on CoDEx-M
  14. Inference Rules in Factual Contexts
  15. Evaluation with Manual Labels
  16. ...and 25 more sections

Figures (3)

  • Figure 1: A hypothetical scenario and its implications, expressed in the language of knowledge graph triples
  • Figure 2: Overview over the different types of facts, given the hypothetical scenario that Elvis Presley is a citizen of Denmark. The green edge (Elvis Presley, speaks, Danish) emerges from adding the blue edge (Elvis Presley, citizen of, Denmark) to the knowledge graph. Purple and orange edges are present in the original KG and unaffected by the scenario. Grey edges are neither present in the original nor the counterfactual knowledge graph.
  • Figure 3: Creation of a hypothetical scenario.