Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Instance-Prototype Affinity Learning for Non-Exemplar Continual Graph Learning

Lei Song, Jiaxing Li, Shihan Guan, Youyong Kong

TL;DR

We address catastrophic forgetting in non-exemplar continual graph learning by leveraging Prototype Contrastive Learning (PCL) and introducing Instance-Prototype Affinity Learning (IPAL). IPAL combines Topology-Integrated Gaussian Prototypes (TIGP) computed with PageRank, Instance-Prototype Affinity Distillation (IPAD) for flexible regularization, and Decision Boundary Perception (DBP) to sharpen inter-class separation, optimized by $\mathcal{L}=\mathcal{L}'_{PCL}+\gamma\mathcal{L}_{AD}$ with $\gamma$ controlling stability-plasticity. Empirical results on four node-classification benchmarks (CS-CL, CoraFull-CL, Arxiv-CL, Reddit-CL) show IPAL outperforms state-of-the-art NECL methods in average performance (AP) while maintaining competitive forgetting (AF), aided by topology-aware prototypes and hard-example mining. The work provides a privacy-preserving, scalable NECGL framework with significant improvements in the plasticity-stability trade-off and offers theoretical support that PCL induces less feature drift than prototype replay. Future work includes adapting IPAL to online/streaming settings and exploring cross-domain generalization for dynamic graphs.

Abstract

Graph Neural Networks (GNN) endure catastrophic forgetting, undermining their capacity to preserve previously acquired knowledge amid the assimilation of novel information. Rehearsal-based techniques revisit historical examples, adopted as a principal strategy to alleviate this phenomenon. However, memory explosion and privacy infringements impose significant constraints on their utility. Non-Exemplar methods circumvent the prior issues through Prototype Replay (PR), yet feature drift presents new challenges. In this paper, our empirical findings reveal that Prototype Contrastive Learning (PCL) exhibits less pronounced drift than conventional PR. Drawing upon PCL, we propose Instance-Prototype Affinity Learning (IPAL), a novel paradigm for Non-Exemplar Continual Graph Learning (NECGL). Exploiting graph structural information, we formulate Topology-Integrated Gaussian Prototypes (TIGP), guiding feature distributions towards high-impact nodes to augment the model's capacity for assimilating new knowledge. Instance-Prototype Affinity Distillation (IPAD) safeguards task memory by regularizing discontinuities in class relationships. Moreover, we embed a Decision Boundary Perception (DBP) mechanism within PCL, fostering greater inter-class discriminability. Evaluations on four node classification benchmark datasets demonstrate that our method outperforms existing state-of-the-art methods, achieving a better trade-off between plasticity and stability.

Instance-Prototype Affinity Learning for Non-Exemplar Continual Graph Learning

TL;DR

We address catastrophic forgetting in non-exemplar continual graph learning by leveraging Prototype Contrastive Learning (PCL) and introducing Instance-Prototype Affinity Learning (IPAL). IPAL combines Topology-Integrated Gaussian Prototypes (TIGP) computed with PageRank, Instance-Prototype Affinity Distillation (IPAD) for flexible regularization, and Decision Boundary Perception (DBP) to sharpen inter-class separation, optimized by with controlling stability-plasticity. Empirical results on four node-classification benchmarks (CS-CL, CoraFull-CL, Arxiv-CL, Reddit-CL) show IPAL outperforms state-of-the-art NECL methods in average performance (AP) while maintaining competitive forgetting (AF), aided by topology-aware prototypes and hard-example mining. The work provides a privacy-preserving, scalable NECGL framework with significant improvements in the plasticity-stability trade-off and offers theoretical support that PCL induces less feature drift than prototype replay. Future work includes adapting IPAL to online/streaming settings and exploring cross-domain generalization for dynamic graphs.

Abstract

Graph Neural Networks (GNN) endure catastrophic forgetting, undermining their capacity to preserve previously acquired knowledge amid the assimilation of novel information. Rehearsal-based techniques revisit historical examples, adopted as a principal strategy to alleviate this phenomenon. However, memory explosion and privacy infringements impose significant constraints on their utility. Non-Exemplar methods circumvent the prior issues through Prototype Replay (PR), yet feature drift presents new challenges. In this paper, our empirical findings reveal that Prototype Contrastive Learning (PCL) exhibits less pronounced drift than conventional PR. Drawing upon PCL, we propose Instance-Prototype Affinity Learning (IPAL), a novel paradigm for Non-Exemplar Continual Graph Learning (NECGL). Exploiting graph structural information, we formulate Topology-Integrated Gaussian Prototypes (TIGP), guiding feature distributions towards high-impact nodes to augment the model's capacity for assimilating new knowledge. Instance-Prototype Affinity Distillation (IPAD) safeguards task memory by regularizing discontinuities in class relationships. Moreover, we embed a Decision Boundary Perception (DBP) mechanism within PCL, fostering greater inter-class discriminability. Evaluations on four node classification benchmark datasets demonstrate that our method outperforms existing state-of-the-art methods, achieving a better trade-off between plasticity and stability.
Paper Structure (20 sections, 1 theorem, 24 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 20 sections, 1 theorem, 24 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Methodology
  5. Topology-Integrated Gaussian Prototypes
  6. Instance-Prototype Affinity Distillation
  7. Decision Boundary Perception
  8. Feature Drift Compensation
  9. Optimization Objective
  10. Experiments
  11. Experimental Setup
  12. Q1: Comparison with the State-of-the-Art
  13. Q2: Ablation Studies and Visualization
  14. Q3: Parameter Analysis
  15. Conclusions and Future Works
  16. ...and 5 more sections

Key Result

Theorem 1

In NECL, assuming Gaussian feature distributions with $\mathcal{F}_{\theta_t}(x_m)\sim \mathcal{P}_t(f)=\mathcal{N}(\mu'_m,\Sigma'_m)$ and $\mathcal{F}_{\theta_{t-1}}(x_{m})\sim \mathcal{P}_{t-1}(f)=\mathcal{N}(\mu_{m},\Sigma_{m})$, where $\Sigma'_m$ and $\Sigma_{m}$ are positive definite, PCL incur

Figures (6)

  • Figure 1: Visualization of feature drift for conventional Prototype Replay (left column) and Prototype Contrastive Learning (right column) on the CS-CL and CoraFull-CL datasets.
  • Figure 2: The overall pipeline of the proposed IPAL framework. Upon the culmination of task $\mathcal{T}_{t-1}$, the TIGP are derived and offline archived in memory buffer $\mathcal{M}$. Following the onset of task $\mathcal{T}_{t}$, online prototypes are dynamically updated and integrated with offline prototypes for PCL. In this regard, IPAD safeguards prior task memory via relational distillation, while DBP ensures the clear demarcation of newly encountered classes. Best viewed in color.
  • Figure 3: Performance heatmaps on three datasets are shown. Top row: Mean-based prototypes for PCL; Middle row: Feature distillation to mitigate feature drift; Bottom row: Our proposed IPAL, which integrates TIGP and IPAD to better balance the trade-off between plasticity and stability.
  • Figure 4: Learning dynamics over the task sequences on CS-CL, CoraFull-CL, Arxiv-CL, and Reddit-CL. The AP is reported on all tasks.
  • Figure 5: Visualization of class distributions on the base task $\mathcal{T}_0$. Each color corresponds to a specific class, and triangles indicate hard examples.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem 1