Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Erase then Rectify: A Training-Free Parameter Editing Approach for Cost-Effective Graph Unlearning

Zhe-Rui Yang, Jindong Han, Chang-Dong Wang, Hao Liu

TL;DR

ETR tackles graph unlearning for GNNs by offering a training-free, two-stage framework. The Erase stage uses neighborhood-aware parameter editing guided by the Fisher Information Matrix to forget unlearned samples and their propagation, while the Rectify stage employs a subgraph-based gradient approximation to boost performance on the remaining data without full retraining. The approach achieves strong model utility, dramatically improved unlearning efficiency, and robust unlearning efficacy across seven datasets, including large-scale graphs, with significant reductions in computation and memory overhead. This work enables scalable, privacy-preserving graph unlearning in real-world deployments by avoiding access to full training data and avoiding complete retraining.

Abstract

Graph unlearning, which aims to eliminate the influence of specific nodes, edges, or attributes from a trained Graph Neural Network (GNN), is essential in applications where privacy, bias, or data obsolescence is a concern. However, existing graph unlearning techniques often necessitate additional training on the remaining data, leading to significant computational costs, particularly with large-scale graphs. To address these challenges, we propose a two-stage training-free approach, Erase then Rectify (ETR), designed for efficient and scalable graph unlearning while preserving the model utility. Specifically, we first build a theoretical foundation showing that masking parameters critical for unlearned samples enables effective unlearning. Building on this insight, the Erase stage strategically edits model parameters to eliminate the impact of unlearned samples and their propagated influence on intercorrelated nodes. To further ensure the GNN's utility, the Rectify stage devises a gradient approximation method to estimate the model's gradient on the remaining dataset, which is then used to enhance model performance. Overall, ETR achieves graph unlearning without additional training or full training data access, significantly reducing computational overhead and preserving data privacy. Extensive experiments on seven public datasets demonstrate the consistent superiority of ETR in model utility, unlearning efficiency, and unlearning effectiveness, establishing it as a promising solution for real-world graph unlearning challenges.

Erase then Rectify: A Training-Free Parameter Editing Approach for Cost-Effective Graph Unlearning

TL;DR

ETR tackles graph unlearning for GNNs by offering a training-free, two-stage framework. The Erase stage uses neighborhood-aware parameter editing guided by the Fisher Information Matrix to forget unlearned samples and their propagation, while the Rectify stage employs a subgraph-based gradient approximation to boost performance on the remaining data without full retraining. The approach achieves strong model utility, dramatically improved unlearning efficiency, and robust unlearning efficacy across seven datasets, including large-scale graphs, with significant reductions in computation and memory overhead. This work enables scalable, privacy-preserving graph unlearning in real-world deployments by avoiding access to full training data and avoiding complete retraining.

Abstract

Graph unlearning, which aims to eliminate the influence of specific nodes, edges, or attributes from a trained Graph Neural Network (GNN), is essential in applications where privacy, bias, or data obsolescence is a concern. However, existing graph unlearning techniques often necessitate additional training on the remaining data, leading to significant computational costs, particularly with large-scale graphs. To address these challenges, we propose a two-stage training-free approach, Erase then Rectify (ETR), designed for efficient and scalable graph unlearning while preserving the model utility. Specifically, we first build a theoretical foundation showing that masking parameters critical for unlearned samples enables effective unlearning. Building on this insight, the Erase stage strategically edits model parameters to eliminate the impact of unlearned samples and their propagated influence on intercorrelated nodes. To further ensure the GNN's utility, the Rectify stage devises a gradient approximation method to estimate the model's gradient on the remaining dataset, which is then used to enhance model performance. Overall, ETR achieves graph unlearning without additional training or full training data access, significantly reducing computational overhead and preserving data privacy. Extensive experiments on seven public datasets demonstrate the consistent superiority of ETR in model utility, unlearning efficiency, and unlearning effectiveness, establishing it as a promising solution for real-world graph unlearning challenges.
Paper Structure (33 sections, 1 theorem, 17 equations, 3 figures, 14 tables, 2 algorithms)

This paper contains 33 sections, 1 theorem, 17 equations, 3 figures, 14 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Graph Unlearning
  4. Training-Free Machine Unlearning
  5. Preliminaries
  6. Notations and Background
  7. Fisher Information Matrix (FIM)
  8. Problem Statement
  9. Methodology
  10. Framework Overview
  11. Graph Unlearning via Parameter Masking
  12. Erase: Neighborhood-Aware Parameter Editing
  13. Rectify: Model Utility Enhancement
  14. Complexity Analysis
  15. Experiments
  16. ...and 18 more sections

Key Result

Theorem 1

For a GNN model, if we approximate the FIM $F$ with its diagonal $diag(F)$, and assume all elements of $diag(F)$ are strictly positive, the mean squared distance between the optimal model parameters trained on $D_r$ and the parameters obtained from (mask), denoted as $Q = \frac{1}{\left| \omega \rig where $c_1$, $c_2$, $c_3$ are constants, and $|\omega|$ denotes the number of parameters.

Figures (3)

  • Figure 1: The framework of ETR: Forgetting the influence of unlearned samples through Erase, followed by enhancing the model performance on the remaining dataset via Rectify.
  • Figure 2: Adversarial edge unlearning on the Cora dataset.
  • Figure 3: Performance with varying hyperparameters.

Theorems & Definitions (2)

  • Theorem 1
  • proof : Proof of Theorem \ref{['theorem']}