Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Continual Learning on Graphs: A Survey

Zonggui Tian, Du Zhang, Hong-Ning Dai

TL;DR

The paper tackles the challenge of learning on evolving graphs by integrating continual learning with graph-based methods to address catastrophic forgetting while pursuing continuous performance improvement (CPI). It introduces a fourfold taxonomy of continual graph learning methods (replay-based, regularization-based, architecture-based, and representation-based) and analyzes their mechanisms for knowledge retention and CPI, including topology-aware regularization, distillation, dynamic architectures, and embedding-level strategies. It surveys replay and generative strategies, regularization and architectural approaches, and representation techniques, linking them to practical applications across NLP, CV, recommender systems, citation networks, bioinformatics, transportation, and finance, while outlining datasets and open issues. The survey also outlines critical open questions on convergence, scalability, robustness, privacy, unsupervised CGL, explainability, and the development of large graph models capable of continual learning, providing guidance for future research directions and real-world deployment. Overall, the work lays a comprehensive foundation for studying CPI in continual graph learning and highlights the theoretical and practical implications for dynamic graph tasks.

Abstract

Recently, continual graph learning has been increasingly adopted for diverse graph-structured data processing tasks in non-stationary environments. Despite its promising learning capability, current studies on continual graph learning mainly focus on mitigating the catastrophic forgetting problem while ignoring continuous performance improvement. To bridge this gap, this article aims to provide a comprehensive survey of recent efforts on continual graph learning. Specifically, we introduce a new taxonomy of continual graph learning from the perspective of overcoming catastrophic forgetting. Moreover, we systematically analyze the challenges of applying these continual graph learning methods in improving performance continuously and then discuss the possible solutions. Finally, we present open issues and future directions pertaining to the development of continual graph learning and discuss how they impact continuous performance improvement.

Continual Learning on Graphs: A Survey

TL;DR

The paper tackles the challenge of learning on evolving graphs by integrating continual learning with graph-based methods to address catastrophic forgetting while pursuing continuous performance improvement (CPI). It introduces a fourfold taxonomy of continual graph learning methods (replay-based, regularization-based, architecture-based, and representation-based) and analyzes their mechanisms for knowledge retention and CPI, including topology-aware regularization, distillation, dynamic architectures, and embedding-level strategies. It surveys replay and generative strategies, regularization and architectural approaches, and representation techniques, linking them to practical applications across NLP, CV, recommender systems, citation networks, bioinformatics, transportation, and finance, while outlining datasets and open issues. The survey also outlines critical open questions on convergence, scalability, robustness, privacy, unsupervised CGL, explainability, and the development of large graph models capable of continual learning, providing guidance for future research directions and real-world deployment. Overall, the work lays a comprehensive foundation for studying CPI in continual graph learning and highlights the theoretical and practical implications for dynamic graph tasks.

Abstract

Recently, continual graph learning has been increasingly adopted for diverse graph-structured data processing tasks in non-stationary environments. Despite its promising learning capability, current studies on continual graph learning mainly focus on mitigating the catastrophic forgetting problem while ignoring continuous performance improvement. To bridge this gap, this article aims to provide a comprehensive survey of recent efforts on continual graph learning. Specifically, we introduce a new taxonomy of continual graph learning from the perspective of overcoming catastrophic forgetting. Moreover, we systematically analyze the challenges of applying these continual graph learning methods in improving performance continuously and then discuss the possible solutions. Finally, we present open issues and future directions pertaining to the development of continual graph learning and discuss how they impact continuous performance improvement.
Paper Structure (75 sections, 1 equation, 7 figures, 5 tables)

This paper contains 75 sections, 1 equation, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Continual Learning
  4. Continual Learning Scenarios
  5. Continual Learning Methods
  6. Evaluation Metrics
  7. Graph Learning
  8. Graphs and Networks
  9. Graph Learning Tasks
  10. Graph Representation Learning
  11. Graph Neural Networks
  12. Continual Graph Learning
  13. Catastrophic Forgetting in Graph Learning
  14. Continual Graph Learning Formalization
  15. Rational of Continual Graph Learning
  16. ...and 60 more sections

Figures (7)

  • Figure 1: Organization of this survey
  • Figure 2: An example of a dynamic graph
  • Figure 3: A taxonomy of continual graph learning methods
  • Figure 4: Replay-based CGL methods
  • Figure 5: Regularization-based CGL methods
  • ...and 2 more figures