Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Machine unlearning through fine-grained model parameters perturbation

Zhiwei Zuo, Zhuo Tang, Kenli Li, Anwitaman Datta

TL;DR

This work introduces SPD-GAN, which subtly perturbs data distribution targeted for unlearning, and proposes novel metrics, namely, the forgetting rate and memory retention rate, which are a novel method for evaluating and quantifying unlearning degree.

Abstract

Machine unlearning techniques, which involve retracting data records and reducing influence of said data on trained models, help with the user privacy protection objective but incur significant computational costs. Weight perturbation-based unlearning is a general approach, but it typically involves globally modifying the parameters. We propose fine-grained Top-K and Random-k parameters perturbed inexact machine unlearning strategies that address the privacy needs while keeping the computational costs tractable. In order to demonstrate the efficacy of our strategies we also tackle the challenge of evaluating the effectiveness of machine unlearning by considering the model's generalization performance across both unlearning and remaining data. To better assess the unlearning effect and model generalization, we propose novel metrics, namely, the forgetting rate and memory retention rate. However, for inexact machine unlearning, current metrics are inadequate in quantifying the degree of forgetting that occurs after unlearning strategies are applied. To address this, we introduce SPD-GAN, which subtly perturbs the distribution of data targeted for unlearning. Then, we evaluate the degree of unlearning by measuring the performance difference of the models on the perturbed unlearning data before and after the unlearning process. By implementing these innovative techniques and metrics, we achieve computationally efficacious privacy protection in machine learning applications without significant sacrifice of model performance. Furthermore, this approach provides a novel method for evaluating the degree of unlearning.

Machine unlearning through fine-grained model parameters perturbation

TL;DR

This work introduces SPD-GAN, which subtly perturbs data distribution targeted for unlearning, and proposes novel metrics, namely, the forgetting rate and memory retention rate, which are a novel method for evaluating and quantifying unlearning degree.

Abstract

Machine unlearning techniques, which involve retracting data records and reducing influence of said data on trained models, help with the user privacy protection objective but incur significant computational costs. Weight perturbation-based unlearning is a general approach, but it typically involves globally modifying the parameters. We propose fine-grained Top-K and Random-k parameters perturbed inexact machine unlearning strategies that address the privacy needs while keeping the computational costs tractable. In order to demonstrate the efficacy of our strategies we also tackle the challenge of evaluating the effectiveness of machine unlearning by considering the model's generalization performance across both unlearning and remaining data. To better assess the unlearning effect and model generalization, we propose novel metrics, namely, the forgetting rate and memory retention rate. However, for inexact machine unlearning, current metrics are inadequate in quantifying the degree of forgetting that occurs after unlearning strategies are applied. To address this, we introduce SPD-GAN, which subtly perturbs the distribution of data targeted for unlearning. Then, we evaluate the degree of unlearning by measuring the performance difference of the models on the perturbed unlearning data before and after the unlearning process. By implementing these innovative techniques and metrics, we achieve computationally efficacious privacy protection in machine learning applications without significant sacrifice of model performance. Furthermore, this approach provides a novel method for evaluating the degree of unlearning.
Paper Structure (29 sections, 18 equations, 9 figures, 2 tables, 2 algorithms)

This paper contains 29 sections, 18 equations, 9 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. User Privacy
  4. Machine Unlearning
  5. Adversarial Learning
  6. Methodology
  7. Problem Definition
  8. Our Unlearning Approaches
  9. Parameter Selection and Perturbation
  10. Unlearning Process
  11. Unlearning Degree Quantification
  12. Time Complexity Analysis
  13. Theorem 1
  14. Theorem 2
  15. Time cost for Top-K parameters calculation
  16. ...and 14 more sections

Figures (9)

  • Figure 1: Random-k unlearning on CIFAR-10, accuracy of $D_{RE}$ and $D_{UL}$ after training for 50 epochs. Baseline is the test accuracy on the source model.
  • Figure 2: Machine Unlearning process using Random-k or Top-K perturbation strategies. When Random-k/Top-K strategies are applied, partial parameters changed (marked pink on unlearning model).
  • Figure 3: SPD-GAN architecture
  • Figure 4: Visualization of images before and after SPD-GAN applied on ResNet18.
  • Figure 5: Accuracy difference between $D_{RE}$ and $D_{UL}$ for Top-K under unlearning data ratio at (a) 5%, (b) 10%, (c) 15% and (d) 20%. The maximum accuracy difference and its corresponding K value is annotated. Such K value is identified as the optimal.
  • ...and 4 more figures