FTF-ER: Feature-Topology Fusion-Based Experience Replay Method for Continual Graph Learning

TL;DR

This work tackles catastrophic forgetting in continual graph learning by redesigning experience replay to leverage both feature-level and global topological information. It introduces Feature-Topology Fusion-based Experience Replay (FTF-ER), which computes feature importance via GraNd and topology importance via a novel Hodge Potential Score, then fuses them into a single sampling criterion. Global topology is obtained through Hodge decomposition on graphs, enabling accurate node ranking without expanding the replay buffer with neighboring nodes, thus reducing storage and runtime costs. Empirical results on four public datasets show state-of-the-art accuracy and efficiency in class-incremental settings, with notable improvements on AA and AF on OGB-Arxiv, while maintaining buffer overhead comparable to topology-agnostic methods. The approach provides a principled and scalable means to utilize comprehensive graph data for continual learning in dynamic environments.

Abstract

Continual graph learning (CGL) is an important and challenging task that aims to extend static GNNs to dynamic task flow scenarios. As one of the mainstream CGL methods, the experience replay (ER) method receives widespread attention due to its superior performance. However, existing ER methods focus on identifying samples by feature significance or topological relevance, which limits their utilization of comprehensive graph data. In addition, the topology-based ER methods only consider local topological information and add neighboring nodes to the buffer, which ignores the global topological information and increases memory overhead. To bridge these gaps, we propose a novel method called Feature-Topology Fusion-based Experience Replay (FTF-ER) to effectively mitigate the catastrophic forgetting issue with enhanced efficiency. Specifically, from an overall perspective to maximize the utilization of the entire graph data, we propose a highly complementary approach including both feature and global topological information, which can significantly improve the effectiveness of the sampled nodes. Moreover, to further utilize global topological information, we propose Hodge Potential Score (HPS) as a novel module to calculate the topological importance of nodes. HPS derives a global node ranking via Hodge decomposition on graphs, providing more accurate global topological information compared to neighbor sampling. By excluding neighbor sampling, HPS significantly reduces buffer storage costs for acquiring topological information and simultaneously decreases training time. Compared with state-of-the-art methods, FTF-ER achieves a significant improvement of 3.6% in AA and 7.1% in AF on the OGB-Arxiv dataset, demonstrating its superior performance in the class-incremental learning setting.

Figures (8)

• Figure 1: FTF-ER workflow: Nodes are selected from each class based on importance scores to create induced subgraphs for the buffer. This enables the model to maintain classification performance across tasks, i.e., continual learning.
• Figure 2: The complete node importance score calculation process for our FTF-ER. In the schematic diagram of Potential, the grayscale of nodes represents their importance, and the edges are only used to indicate the comparative results of importance. In the Sample Score stage, before mixing the two scores, they need to be normalized separately as shown in Eq. (\ref{['equ:norm']}).
• Figure 3: Acquiring topological information by random node sampling, SSM SSM with 1-hop, 2-hop neighbor sampling, and FTF-ER (ours). Green nodes indicate the ones selected to be added to the buffer, and bold solid lines represent the information flow between nodes during the sampling process.
• Figure 4: Evolution of the AA throughout the learning process on the task sequences of four datasets.
• Figure 5: Visualization of the performance matrices of our FTF-ER method across four datasets.

Theorems & Definitions (14)

• definition 1: Hodge Potential Score
• definition 2: Edge Flows
• definition 3: Gradient Operator
• definition 4: Negative Divergence Operator
• definition 5: Graph Laplacian Operator
• definition 6: Hodge-Laplace operator
• theorem 1: Hodge Decomposition Theorem
• definition 7: Hodge Potential Score
• definition 8: Edge Flows
• definition 9: Gradient Operator