Editing Language Model-based Knowledge Graph Embeddings

TL;DR

<3-5 sentence high-level summary> This work introduces a formal task for editing language model–based knowledge graph embeddings to support post-deployment updates, defining EDIT for corrections and ADD for new facts. It constructs four challenging datasets and establishes robust evaluation metrics to measure reliability, locality, and efficiency of edits. The authors propose KGEditor, a hypernetwork-guided extra FFN layer that enables data-efficient updates with limited parameter changes, outperforming external editors (KE, MEND) and CALINET while preserving existing knowledge. Prompt-tuned KGEs generally yield better editing performance than fine-tuned ones, and the approach demonstrates scalable editing as the number of edited facts grows. The work provides code and datasets to foster further research in dynamic knowledge-augmented language models.

Abstract

Recently decades have witnessed the empirical success of framing Knowledge Graph (KG) embeddings via language models. However, language model-based KG embeddings are usually deployed as static artifacts, making them difficult to modify post-deployment without re-training after deployment. To address this issue, we propose a new task of editing language model-based KG embeddings in this paper. This task is designed to facilitate rapid, data-efficient updates to KG embeddings without compromising the performance of other aspects. We build four new datasets: E-FB15k237, A-FB15k237, E-WN18RR, and A-WN18RR, and evaluate several knowledge editing baselines demonstrating the limited ability of previous models to handle the proposed challenging task. We further propose a simple yet strong baseline dubbed KGEditor, which utilizes additional parametric layers of the hypernetwork to edit/add facts. Our comprehensive experimental results reveal that KGEditor excels in updating specific facts without impacting the overall performance, even when faced with limited training resources. Code and datasets are available in https://github.com/zjunlp/PromptKG/tree/main/deltaKG.

Figures (6)

• Figure 1: Top: Illustration of the EDIT task. We edit the wrong fact knowledge stored in the KG embeddings. Bottom: Illustration of the ADD task. We add brand-new knowledge into the model without re-training.
• Figure 2: Data Construction Process. Step 1: Randomly disrupt the triples from the existing knowledge graph to produce a corrupted dataset. Step 2: Employ this dataset to fine-tune a pre-existing pre-trained KGE model, yielding a model needing editing. Step 3: Filtering, we reassess the data with the pre-trained KGE model, accurately sorting the correctly labeled data into the L-Test dataset and allocating the mislabeled data to a designated dataset for correction.
• Figure 3: The introduction of baselines and KGEditor. The external model-based editors (a) utilize a hyper external network to obtain the parameters’ shift and add to the original model parameters for editing (replacing the origin entity $y$ with the alternative entity $a$). The additional parameter-based editors (b) rectify the erroneous knowledge stored in FFN by adjusting its predicted distributions from $\operatorname{FFN}(H;W)$ to $\operatorname{FFN}^{\prime}(H)$. KGEditor (c) utilize a hyper external network to update the knowledge in FFN (Top right: EDIT, Bottom right: ADD).
• Figure 4: Left: $ER_{roc}$ and Right: $RK_{roc}$. The performance of different KGE initialization with different models.
• Figure 5: Left: the variation of Succ@1 value with $n$ edits. All models drop when $n$ edits become larger. Right: the RK@3 score drops when $n$ changes, KE and KGEditor can remain stable performance, while MEND decreases.