GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation

TL;DR

GraphKeeper tackles Graph Domain-Incremental Learning (Domain-IL) by disentangling domain representations and preserving stable decisions across evolving graph domains. It introduces domain-specific PEFT (LoRA-based) with intra- and inter-domain disentanglement to prevent embedding shifts, and a deviation-free knowledge preservation via ridge regression to keep decision boundaries consistent. A domain-aware distribution discrimination module enables precise embeddings for graphs with unobservable domains. Empirical results show state-of-the-art performance with strong robustness and compatibility with representative GFMs, underscoring its practical impact for scalable, continual graph knowledge bases.

Abstract

Graph incremental learning (GIL), which continuously updates graph models by sequential knowledge acquisition, has garnered significant interest recently. However, existing GIL approaches focus on task-incremental and class-incremental scenarios within a single domain. Graph domain-incremental learning (Domain-IL), aiming at updating models across multiple graph domains, has become critical with the development of graph foundation models (GFMs), but remains unexplored in the literature. In this paper, we propose Graph Domain-Incremental Learning via Knowledge Dientanglement and Preservation (GraphKeeper), to address catastrophic forgetting in Domain-IL scenario from the perspectives of embedding shifts and decision boundary deviations. Specifically, to prevent embedding shifts and confusion across incremental graph domains, we first propose the domain-specific parameter-efficient fine-tuning together with intra- and inter-domain disentanglement objectives. Consequently, to maintain a stable decision boundary, we introduce deviation-free knowledge preservation to continuously fit incremental domains. Additionally, for graphs with unobservable domains, we perform domain-aware distribution discrimination to obtain precise embeddings. Extensive experiments demonstrate the proposed GraphKeeper achieves state-of-the-art results with 6.5%~16.6% improvement over the runner-up with negligible forgetting. Moreover, we show GraphKeeper can be seamlessly integrated with various representative GFMs, highlighting its broad applicative potential.

Figures (9)

• Figure 1: An illustration of traditional GIL (i.e., Task-IL and Class-IL) and Domain-IL scenarios within our scope studied in this paper.
• Figure 2: Performance of SSM zhang2022sparsified, a representative GIL method, in Class-IL scenario and the more challenging Domain-IL scenario.
• Figure 3: The overall framework of GraphKeeper. (1) The Multi-domain Graph Disentanglement isolates parameters of different graph domains through the domain-specific graph PEFT to prevent embedding shifts, and disentangles embeddings both intra- and inter-domain to prevent confusion. (2) The Deviation-Free Knowledge Preservation continuously fits incremental graph domains while maintaining a stable decision boundary without deviations. (3) The Domain-aware Distribution Discrimination matchs graphs with unobservable domain to prototypes of previous domains. Then our method embeds them with corresponding domain-specific PEFT module, and make predictions.
• Figure 4: The ablation study results.
• Figure 5: Hyperparameter analysis.

Theorems & Definitions (1)

• proof