AL-GNN: Privacy-Preserving and Replay-Free Continual Graph Learning via Analytic Learning

TL;DR

This work tackles the challenge of continual graph learning under privacy constraints by proposing AL-GNN, a framework that eliminates backpropagation and replay buffers. It formulates learning as recursive least-squares updates, maintaining a regularized feature autocorrelation matrix to enable one-pass, memory-efficient updates and to preserve past knowledge without storing historical data. The method achieves competitive or superior performance across six graph benchmarks, with substantial improvements in forgetting mitigation and training time, while offering strong privacy guarantees. A fixed pre-trained backbone underpins the approach, highlighting a trade-off between stability and adaptability that may guide future enhancements.

Abstract

Continual graph learning (CGL) aims to enable graph neural networks to incrementally learn from a stream of graph structured data without forgetting previously acquired knowledge. Existing methods particularly those based on experience replay typically store and revisit past graph data to mitigate catastrophic forgetting. However, these approaches pose significant limitations, including privacy concerns, inefficiency. In this work, we propose AL GNN, a novel framework for continual graph learning that eliminates the need for backpropagation and replay buffers. Instead, AL GNN leverages principles from analytic learning theory to formulate learning as a recursive least squares optimization process. It maintains and updates model knowledge analytically through closed form classifier updates and a regularized feature autocorrelation matrix. This design enables efficient one pass training for each task, and inherently preserves data privacy by avoiding historical sample storage. Extensive experiments on multiple dynamic graph classification benchmarks demonstrate that AL GNN achieves competitive or superior performance compared to existing methods. For instance, it improves average performance by 10% on CoraFull and reduces forgetting by over 30% on Reddit, while also reducing training time by nearly 50% due to its backpropagation free design.

Figures (10)

• Figure 1: AL-GNN starts with the basic training steps: (a) GCN is trained for $m$ epochs using a BP-based iterative method on a dataset of half the classes. (b) The original output layer is replaced by feg, and feature extraction is enhanced by expanding the dimension. Only one epoch of analytical alignment is performed on the same dataset to calculate $W_0$ and $R_0$. (c–d) The AL step is performed recursively, using only the dataset of the current stage (training for one epoch) and the correlation matrix encrypted using historical information.
• Figure 2: Task-wise performance (AP) over incremental tasks across six graph datasets. The proposed AL-GNN consistently outperforms the baseline method in final AP across all settings, and maintains stable performance under long task sequences (e.g., Corafull, Reddit, Ogbn-arxiv). In the supplementary material further analyzes multi-class increments.
• Figure 3: Task-wise performance matrix across different methods and datasets. Each heatmap visualizes the classification accuracy of each task during continual training. The cell at row $i$, column $j$ indicates the model’s accuracy on task $\mathcal{G}_j$ after training on task $\mathcal{G}_i$. Brighter colors indicate higher accuracy, and darker colors suggest performance degradation. Diagonal intensity reflects current task learning, the cells in the same column change color less as they go down, which means they forget less.
• Figure 4: The impact of FEG on incremental learning performance
• Figure 5: The impact of $\gamma$ on incremental learning performance