EntailE: Introducing Textual Entailment in Commonsense Knowledge Graph Completion

TL;DR

EntailE addresses sparsity in commonsense knowledge graph completion by introducing textual entailment signals between node contexts. It leverages a finetuned NLI transformer to identify entailed nodes and densify the CSKG via synthetic triplets, while an entity contrast module mitigates noise during embedding learning. The model uses a ConvTransE decoder with node representations initialized from a BERT based embedding of node context, and trains with a joint loss including synthetic triplet and contrastive terms. Experiments on CN-82K and ATOMIC show consistent improvements in both transductive and inductive settings, highlighting the value of textual entailment for commonsense knowledge graphs.

Abstract

Commonsense knowledge graph completion is a new challenge for commonsense knowledge graph construction and application. In contrast to factual knowledge graphs such as Freebase and YAGO, commonsense knowledge graphs (CSKGs; e.g., ConceptNet) utilize free-form text to represent named entities, short phrases, and events as their nodes. Such a loose structure results in large and sparse CSKGs, which makes the semantic understanding of these nodes more critical for learning rich commonsense knowledge graph embedding. While current methods leverage semantic similarities to increase the graph density, the semantic plausibility of the nodes and their relations are under-explored. Previous works adopt conceptual abstraction to improve the consistency of modeling (event) plausibility, but they are not scalable enough and still suffer from data sparsity. In this paper, we propose to adopt textual entailment to find implicit entailment relations between CSKG nodes, to effectively densify the subgraph connecting nodes within the same conceptual class, which indicates a similar level of plausibility. Each node in CSKG finds its top entailed nodes using a finetuned transformer over natural language inference (NLI) tasks, which sufficiently capture textual entailment signals. The entailment relation between these nodes are further utilized to: 1) build new connections between source triplets and entailed nodes to densify the sparse CSKGs; 2) enrich the generalization ability of node representations by comparing the node embeddings with a contrastive loss. Experiments on two standard CSKGs demonstrate that our proposed framework EntailE can improve the performance of CSKG completion tasks under both transductive and inductive settings.

Figures (5)

• Figure 1: An example of densifying ATOMIC with nodes which are semantically related. Solid lines indicate the original edges and dashed lines are the densified edges using commonsense plausibility.
• Figure 2: Conceptual abstraction for finding plausibility consistent context node in ATOMIC. Texts in Italic type are matched nodes in ATOMIC.
• Figure 3: The overview framework for EntailE. CSKGs are densified by the synthetic triplets. Entity contrast module groups the representation of source nodes with its entailed nodes. The triplet learning loss and entity contrast loss are trained alternatively.
• Figure 4: Percentage of inductive CSKG nodes in the valid split covered by the entailed nodes of training split in CN-82K. $k$ is the number of entailed nodes for each node in the training split.
• Figure 5: Percentage of inductive ATOMIC nodes in the valid split covered by the entailed nodes of training split in ATOMIC. $k$ is the number of entailed nodes for each node in the training split.