Dynamic Graph Unlearning: A General and Efficient Post-Processing Method via Gradient Transformation

TL;DR

This paper tackles the privacy challenge posed by dynamic graph data in DGNNs, proposing a general post-processing method, Gradient Transformation, to realize dynamic graph unlearning without architectural changes. It formalizes unlearning for continuous-time dynamic graphs, computes an initial gradient from unlearning requests, and learns a gradient-transforming function to produce a parameter update that mimics retraining on the remaining data. The approach combines a two-layer MLP-Mixer with a multi-term unlearning loss to balance remaining-data fidelity, unlearning accuracy, and generalization, achieving strong results across six real-world datasets and two backbone DGNNs. Empirically, Gradient Transformation delivers competitive or superior test-time performance, significantly improved unlearning effectiveness, and substantial speed-ups (up to 32x) for future unlearning scenarios, illustrating its practical impact for privacy-preserving DGNN deployments. The method is architecture-agnostic, scalable in practice, and demonstrates potential for broader applicability beyond the tested models and tasks.

Abstract

Dynamic graph neural networks (DGNNs) have emerged and been widely deployed in various web applications (e.g., Reddit) to serve users (e.g., personalized content delivery) due to their remarkable ability to learn from complex and dynamic user interaction data. Despite benefiting from high-quality services, users have raised privacy concerns, such as misuse of personal data (e.g., dynamic user-user/item interaction) for model training, requiring DGNNs to ``forget'' their data to meet AI governance laws (e.g., the ``right to be forgotten'' in GDPR). However, current static graph unlearning studies cannot \textit{unlearn dynamic graph elements} and exhibit limitations such as the model-specific design or reliance on pre-processing, which disenable their practicability in dynamic graph unlearning. To this end, we study the dynamic graph unlearning for the first time and propose an effective, efficient, general, and post-processing method to implement DGNN unlearning. Specifically, we first formulate dynamic graph unlearning in the context of continuous-time dynamic graphs, and then propose a method called Gradient Transformation that directly maps the unlearning request to the desired parameter update. Comprehensive evaluations on six real-world datasets and state-of-the-art DGNN backbones demonstrate its effectiveness (e.g., limited drop or obvious improvement in utility) and efficiency (e.g., 7.23$\times$ speed-up) advantages. Additionally, our method has the potential to handle future unlearning requests with significant performance gains (e.g., 32.59$\times$ speed-up).

Figures (5)

• Figure 1: An overview of the unlearning of DGNNs. (1) In the upper half, given a dynamic graph $S$ and a DGNN $f$, the model developer obtains the optimal $f_{\theta^{*}}$, which makes accurate predictions on both training ($t\leq 8$) and test ($t>8$) data. (2) The lower half illustrates the ideal unlearning process. Upon receiving the unlearning request $S_{ul}$, the model developer removes $S_{ul}$ from $S$ and retrains $f$ from scratch to obtain the ideal DGNN $f_{\theta_{ul}^{*}}$. However, retraining from scratch requires huge source costs (e.g., time and computational source). (3) To this end, this paper aims to devise an effective and efficient unlearning method $\mathcal{U}$ to approximate the parameter obtained from retraining, as indicated by the green arrows.
• Figure 2: The overview of our Gradient Transformation method.
• Figure 3: The time cost comparison between our Gradient Transformation method and baseline methods. The numerical values on the bars (e.g., $6.20\times$) indicate the degree of acceleration relative to the retraining approach.
• Figure 4: An overview of the complex interaction between $S_{re}$ and $S_{ul}$. (1) In the upper half, given a dynamic graph $S$ (maximum event time $T_{S}=8$) and a DGNN $f$, we use an algorithm $\mathcal{A}_{f}$ to obtain the optimal $f_{\theta^{*}}$, which can make accurate predictions on both training ($t\leq T_{S}$) and test ($t>T_{S}$) data. (2)Green/blue/brown arrows indicate the spatial and temporal neighbors of node $v_i$/$v_j$/$v_d$ in the last 2 historical time points. As shown in the middle column, DGNNs generally use the derived spatial-temporal subgraphs to obtain the node embedding. By combining the embedding of two nodes, $f$ can predict whether there is an edge between them at specific time points. (3) In the lower half, upon receiving the unlearning request $S_{ul}$, an unlearning method aims to approximate the parameter obtained from retraining $f$ with $S_{ul}$. (4) The dashed box on the left indicates the change in training data. The middle dashed box identifies the changed spatial-temporal subgraph of node $v_j$, potentially resulting in changed embedding at time $t=8$. The right dashed box highlights the desired prediction change from the perspective of unlearning.
• Figure 5: An illustration of the potential trade-off between model generalization and unlearning requests on the training dataset. See Appx. \ref{['sec:appendix:generalisation']} for more details.