Continual Learning on Graphs: Challenges, Solutions, and Opportunities

TL;DR

This survey addresses continual learning on graphs (CGL) by outlining the unique challenges posed by evolving graph structures and the need to avoid catastrophic forgetting. It classifies problem settings into task-, domain-, and class-incremental learning, and distinguishes node/edge-level from graph-level tasks, then surveys regularization-, memory replay-, and parameter isolation-based methods, including adaptations of traditional CL techniques. The paper also surveys benchmarks and datasets, evaluation metrics, and future directions such as topology preservation, task-free learning, and knowledge transfer, while connecting CGL to graph foundation models. By providing a unified framework and up-to-date resources, it aims to accelerate systematic progress and practical adoption of CGL across diverse graph domains.

Abstract

Continual learning on graph data has recently attracted paramount attention for its aim to resolve the catastrophic forgetting problem on existing tasks while adapting the sequentially updated model to newly emerged graph tasks. While there have been efforts to summarize progress on continual learning research over Euclidean data, e.g., images and texts, a systematic review of progress in continual learning on graphs, a.k.a, continual graph learning (CGL) or lifelong graph learning, is still demanding. Graph data are far more complex in terms of data structures and application scenarios, making CGL task settings, model designs, and applications extremely challenging. To bridge the gap, we provide a comprehensive review of existing continual graph learning (CGL) algorithms by elucidating the different task settings and categorizing the existing methods based on their characteristics. We compare the CGL methods with traditional continual learning techniques and analyze the applicability of the traditional continual learning techniques to CGL tasks. Additionally, we review the benchmark works that are crucial to CGL research. Finally, we discuss the remaining challenges and propose several future directions. We will maintain an up-to-date GitHub repository featuring a comprehensive list of CGL algorithms, accessible at https://github.com/UConn-DSIS/Survey-of-Continual-Learning-on-Graphs.

Figures (6)

• Figure 1: Illustration of the different incremental settings.
• Figure 2: Timeline of the development of different CGL techniques.
• Figure 3: Topology preservation design in TWP liu2021overcoming.
• Figure 4: Pipelines of two representative memory replay based techniques. (a) ER-GNN zhou2021overcoming stores single nodes. (b) SSM zhang2022sparsified stores sparsified computation subgraphs.
• Figure 5: Pipelines of HPNs 9808404 stores sparsified computation subgraphs.