Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks

Xuran Li, Jingyi Wang, Xiaohan Yuan, Peixin Zhang

TL;DR

This work addresses certifiable unlearning in neural networks by introducing PRUNE, a patch-based repair framework that injects a minimal patch into the original model to force forgetting of targeted data. The patch comprises a confusion sub-network that redirects predictions and a support sub-network that confines side effects, yielding a provable forgetting guarantee while preserving performance on remaining data. For multiple data points, the method selects representative instances via clustering and iteratively refines patches, achieving desired forgetting levels with bounded impact. Experiments on four datasets show effective forgetting with competitive efficiency, enabling auditable data erasure suitable for regulatory requirements.

Abstract

It is often desirable to remove (a.k.a. unlearn) a specific part of the training data from a trained neural network model. A typical application scenario is to protect the data holder's right to be forgotten, which has been promoted by many recent regulation rules. Existing unlearning methods involve training alternative models with remaining data, which may be costly and challenging to verify from the data holder or a thirdparty auditor's perspective. In this work, we provide a new angle and propose a novel unlearning approach by imposing carefully crafted "patch" on the original neural network to achieve targeted "forgetting" of the requested data to delete. Specifically, inspired by the research line of neural network repair, we propose to strategically seek a lightweight minimum "patch" for unlearning a given data point with certifiable guarantee. Furthermore, to unlearn a considerable amount of data points (or an entire class), we propose to iteratively select a small subset of representative data points to unlearn, which achieves the effect of unlearning the whole set. Extensive experiments on multiple categorical datasets demonstrates our approach's effectiveness, achieving measurable unlearning while preserving the model's performance and being competitive in efficiency and memory consumption compared to various baseline methods.

PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks

TL;DR

This work addresses certifiable unlearning in neural networks by introducing PRUNE, a patch-based repair framework that injects a minimal patch into the original model to force forgetting of targeted data. The patch comprises a confusion sub-network that redirects predictions and a support sub-network that confines side effects, yielding a provable forgetting guarantee while preserving performance on remaining data. For multiple data points, the method selects representative instances via clustering and iteratively refines patches, achieving desired forgetting levels with bounded impact. Experiments on four datasets show effective forgetting with competitive efficiency, enabling auditable data erasure suitable for regulatory requirements.

Abstract

It is often desirable to remove (a.k.a. unlearn) a specific part of the training data from a trained neural network model. A typical application scenario is to protect the data holder's right to be forgotten, which has been promoted by many recent regulation rules. Existing unlearning methods involve training alternative models with remaining data, which may be costly and challenging to verify from the data holder or a thirdparty auditor's perspective. In this work, we provide a new angle and propose a novel unlearning approach by imposing carefully crafted "patch" on the original neural network to achieve targeted "forgetting" of the requested data to delete. Specifically, inspired by the research line of neural network repair, we propose to strategically seek a lightweight minimum "patch" for unlearning a given data point with certifiable guarantee. Furthermore, to unlearn a considerable amount of data points (or an entire class), we propose to iteratively select a small subset of representative data points to unlearn, which achieves the effect of unlearning the whole set. Extensive experiments on multiple categorical datasets demonstrates our approach's effectiveness, achieving measurable unlearning while preserving the model's performance and being competitive in efficiency and memory consumption compared to various baseline methods.
Paper Structure (10 sections, 5 theorems, 7 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 10 sections, 5 theorems, 7 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methodology
  3. Problem Definition
  4. The PRUNE Framework
  5. Experiments
  6. Experimental Setup
  7. Utility Guarantee
  8. Efficiency Comparison
  9. Hyper-parameter Tuning
  10. Conclusion

Key Result

Theorem 1

Given a neural network model $M_{\mathcal{D}}=M_p\oplus M_c$ trained on $\mathcal{D}$, and a sample data $x_u$ to unlearn, it is guaranteed that we can construct a patch network $c_S$ for $M_c$ and obtain an unlearned model $M_U=M_p\oplus (M_c+c_S)$ such that: 1) $M_U(x_u)\neq M_{\mathcal{D}}(x_u)$;

Figures (4)

  • Figure 1: The framework of PRUNE.
  • Figure 2: The efficiency of different methods to unlearn on multiple datasets.
  • Figure 3: The convergence of PRUNE w.r.t the number of centroids $K$ ($|\mathcal{D}_U|=100$).
  • Figure 4: The convergence of PRUNE w.r.t the scale of $\mathcal{D}_U$ ($K=4$).

Theorems & Definitions (5)

  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Theorem 2