Learning to Evolve: Bayesian-Guided Continual Knowledge Graph Embedding

TL;DR

BAKE addresses the challenge of continual knowledge graph embedding (CKGE) for evolving graphs by reframing CKGE as sequential Bayesian inference, where the posterior from each snapshot serves as the prior for the next. It couples a Bayesian-guided knowledge evolution with a continual clustering regularizer to actively preserve past knowledge while allowing controlled adaptation to new facts, using a joint objective that blends KGE loss, posterior regularization, and semantic drift control. Empirically, BAKE achieves state-of-the-art performance across eight CKGE benchmarks, with robust forward transfer and reduced embedding drift over time. This principled approach enables scalable, memory-aware knowledge accumulation in dynamical knowledge graphs with practical impact for time-sensitive reasoning and long-term semantic consistency.

Abstract

As social media and the World Wide Web become hubs for information dissemination, effectively organizing and understanding the vast amounts of dynamically evolving Web content is crucial. Knowledge graphs (KGs) provide a powerful framework for structuring this information. However, the rapid emergence of new hot topics, user relationships, and events in social media renders traditional static knowledge graph embedding (KGE) models rapidly outdated. Continual Knowledge Graph Embedding (CKGE) aims to address this issue, but existing methods commonly suffer from catastrophic forgetting, whereby older, but still valuable, information is lost when learning new knowledge (such as new memes or trending events). This means the model cannot effectively learn the evolution of the data. We propose a novel CKGE framework, BAKE. Unlike existing methods, BAKE formulates CKGE as a sequential Bayesian inference problem and utilizes the Bayesian posterior update principle as a natural continual learning strategy. This principle is insensitive to data order and provides theoretical guarantees to preserve prior knowledge as much as possible. Specifically, we treat each batch of new data as a Bayesian update to the model's prior. By maintaining the posterior distribution, the model effectively preserves earlier knowledge even as it evolves over multiple snapshots. Furthermore, to constrain the evolution of knowledge across snapshots, we introduce a continual clustering method that maintains the compact cluster structure of entity embeddings through a regularization term, ensuring semantic consistency while allowing controlled adaptation to new knowledge. We conduct extensive experiments on multiple CKGE benchmarks, which demonstrate that BAKE achieves the top performance in the vast majority of cases compared to existing approaches.

Figures (5)

• Figure 1: TKG uses a timestamp quadruple to explicitly annotate temporal information, while CKG decomposes the knowledge graph into time-unmarked evolutionary snapshots, focusing on the continuous updating and migration of knowledge.
• Figure 2: An overview of our proposed BAKE framework.
• Figure 3: The change of model performance with the evolution of snapshots under ENTITY and FB_CKGE datasets.
• Figure 4: Parameter Sensitivity of BAKE to Cluster Number K under Different Snapshots on the ENTITY, GRAPH_LOWER, FB_CKGE, and WN_CKGE Datasets.
• Figure 5: Knowledge transfer experiments on eight datasets. BAKE w/o B represents the use of knowledge distillation to retain the original knowledge without the guidance of Bayesian principles, and BAKE w/o C represents the use of no continual clustering method.