A Contextualized BERT model for Knowledge Graph Completion

TL;DR

The paper tackles Knowledge Graph Completion (KGC) by predicting missing tail entities using contextual information from neighboring entities and relations. It introduces CAB-KGC, a contextualized BERT-based model that computes head context $H_c$ and relation context $R_c$ and feeds them with the head $h$ and relation $r$ into a classifier to model $P(t|h,r)$; the input sequence is [CLS] h, $H_c$ [SEP] r, $R_c$. This approach avoids reliance on entity descriptions and negative sampling, reducing computation. On FB15k-237 and WN18RR, CAB-KGC achieves state-of-the-art performance with notable Hits@1 improvements of 5.3% and 4.88%, respectively, validating the effectiveness of contextual cues.

Abstract

Knowledge graphs (KGs) are valuable for representing structured, interconnected information across domains, enabling tasks like semantic search, recommendation systems and inference. A pertinent challenge with KGs, however, is that many entities (i.e., heads, tails) or relationships are unknown. Knowledge Graph Completion (KGC) addresses this by predicting these missing nodes or links, enhancing the graph's informational depth and utility. Traditional methods like TransE and ComplEx predict tail entities but struggle with unseen entities. Textual-based models leverage additional semantics but come with high computational costs, semantic inconsistencies, and data imbalance issues. Recent LLM-based models show improvement but overlook contextual information and rely heavily on entity descriptions. In this study, we introduce a contextualized BERT model for KGC that overcomes these limitations by utilizing the contextual information from neighbouring entities and relationships to predict tail entities. Our model eliminates the need for entity descriptions and negative triplet sampling, reducing computational demands while improving performance. Our model outperforms state-of-the-art methods on standard datasets, improving Hit@1 by 5.3% and 4.88% on FB15k-237 and WN18RR respectively, setting a new benchmark in KGC.

Figures (1)

• Figure 1: A concise view of the CAB-KGC Method. Box on the left shows head context $H_c$ calculation; the middle one shows relationship context $R_c$ calculation. $H_c$ and $R_c$ are then fed into the model pipeline shown on the right side.