GNNDelete: A General Strategy for Unlearning in Graph Neural Networks

TL;DR

GNNDelete introduces a model-agnostic, layer-wise deletion operator for graph unlearning in GNNs. By enforcing Deleted Edge Consistency and Neighborhood Influence, and learning small per-layer deletion weights $W_D^l$, it deletes targeted graph elements while preserving remaining knowledge, without retraining from scratch. Empirical results across seven real-world graphs show substantial improvements over baselines in edge-, node-, and feature-deletion tasks, along with notable time and space efficiency. The work demonstrates practical applicability to a range of GNN architectures and even other graph-representations, underscoring its potential as a general strategy for graph unlearning in privacy- and accuracy-sensitive settings.

Abstract

Graph unlearning, which involves deleting graph elements such as nodes, node labels, and relationships from a trained graph neural network (GNN) model, is crucial for real-world applications where data elements may become irrelevant, inaccurate, or privacy-sensitive. However, existing methods for graph unlearning either deteriorate model weights shared across all nodes or fail to effectively delete edges due to their strong dependence on local graph neighborhoods. To address these limitations, we introduce GNNDelete, a novel model-agnostic layer-wise operator that optimizes two critical properties, namely, Deleted Edge Consistency and Neighborhood Influence, for graph unlearning. Deleted Edge Consistency ensures that the influence of deleted elements is removed from both model weights and neighboring representations, while Neighborhood Influence guarantees that the remaining model knowledge is preserved after deletion. GNNDelete updates representations to delete nodes and edges from the model while retaining the rest of the learned knowledge. We conduct experiments on seven real-world graphs, showing that GNNDelete outperforms existing approaches by up to 38.8% (AUC) on edge, node, and node feature deletion tasks, and 32.2% on distinguishing deleted edges from non-deleted ones. Additionally, GNNDelete is efficient, taking 12.3x less time and 9.3x less space than retraining GNN from scratch on WordNet18.

Figures (3)

• Figure 1: a. Illustration of Deleted Edge Consistency: It suggests that the predicted probability of deleted edges after unlearning should be random, such that it looks like the deleted data was not used for training before. b. Illustration of Neighborhood Influence: It implies that an appropriate unlearning should not change the representations of the local neighborhood (nodes in the subgraph, not nodes themselves ) to maintain the original causality. c. Overview of GNNDelete: Given a trained GNN model and edge deletion request, GNNDelete outputs unlearned representations efficiently by only learning a small deletion operator $W_D$. It also ensures representation quality by minimizing a loss function that satisfies the two key properties proposed above.
• Figure 2: Comparison of efficiency on three datasets (PubMed, CS, and OGB-Collab). We plot the retraining approach in solid lines, general unlearning methods in dotted lines, and graph unlearning methods in dash-dotted lines. Results show that GNNDelete scales better than existing graph unlearning methods, as its execution time is consistently lower than other methods, especially for larger graphs.
• Figure 3: Ablation study on the interplay of Deleted Edge Consistency and Neighborhood Influence property. Unlearning task: 2.5% edge deletion. Evaluation: link prediction. Dataset: DBLP. The gap is calculated as: $|\textrm{AUROC}(\mathcal{E}_t) - \textrm{AUROC}(\mathcal{E}_d)|$. Best overall deletion performance is achieved for $\lambda=0.5$, indicating that both properties are necessary to successfully delete information from the GNN model while minimizing negative effects on overall model performance.

Theorems & Definitions (3)

• Definition 1: Deleted Edge Consistency
• Definition 2: Neighborhood Influence
• Theorem 1