Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services

Hongsheng Hu, Shuo Wang, Jiamin Chang, Haonan Zhong, Ruoxi Sun, Shuang Hao, Haojin Zhu, Minhui Xue

TL;DR

This study uncovers an underexplored gap between unlearning and contemporary MLaaS, highlighting the need for careful considerations in balancing data unlearning, model utility, and security.

Abstract

The right to be forgotten requires the removal or "unlearning" of a user's data from machine learning models. However, in the context of Machine Learning as a Service (MLaaS), retraining a model from scratch to fulfill the unlearning request is impractical due to the lack of training data on the service provider's side (the server). Furthermore, approximate unlearning further embraces a complex trade-off between utility (model performance) and privacy (unlearning performance). In this paper, we try to explore the potential threats posed by unlearning services in MLaaS, specifically over-unlearning, where more information is unlearned than expected. We propose two strategies that leverage over-unlearning to measure the impact on the trade-off balancing, under black-box access settings, in which the existing machine unlearning attacks are not applicable. The effectiveness of these strategies is evaluated through extensive experiments on benchmark datasets, across various model architectures and representative unlearning approaches. Results indicate significant potential for both strategies to undermine model efficacy in unlearning scenarios. This study uncovers an underexplored gap between unlearning and contemporary MLaaS, highlighting the need for careful considerations in balancing data unlearning, model utility, and security.

A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services

TL;DR

This study uncovers an underexplored gap between unlearning and contemporary MLaaS, highlighting the need for careful considerations in balancing data unlearning, model utility, and security.

Abstract

The right to be forgotten requires the removal or "unlearning" of a user's data from machine learning models. However, in the context of Machine Learning as a Service (MLaaS), retraining a model from scratch to fulfill the unlearning request is impractical due to the lack of training data on the service provider's side (the server). Furthermore, approximate unlearning further embraces a complex trade-off between utility (model performance) and privacy (unlearning performance). In this paper, we try to explore the potential threats posed by unlearning services in MLaaS, specifically over-unlearning, where more information is unlearned than expected. We propose two strategies that leverage over-unlearning to measure the impact on the trade-off balancing, under black-box access settings, in which the existing machine unlearning attacks are not applicable. The effectiveness of these strategies is evaluated through extensive experiments on benchmark datasets, across various model architectures and representative unlearning approaches. Results indicate significant potential for both strategies to undermine model efficacy in unlearning scenarios. This study uncovers an underexplored gap between unlearning and contemporary MLaaS, highlighting the need for careful considerations in balancing data unlearning, model utility, and security.
Paper Structure (24 sections, 13 equations, 10 figures, 13 tables, 1 algorithm)

This paper contains 24 sections, 13 equations, 10 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work and Threat Model
  3. Related Work
  4. Our Threat Model
  5. Difference from Existing Threats to Machine Unlearning
  6. Methodology of Malicious Unlearning
  7. Problem Statement
  8. Blending as Naive Over-unlearning
  9. Pushing as Advanced Over-unlearning
  10. Experimental Settings
  11. Datasets and Models
  12. Unlearning Settings and Benchmarks
  13. Evaluation
  14. Effectiveness of Blending Method
  15. Effectiveness of Pushing-I and Pushing-II
  16. ...and 9 more sections

Figures (10)

  • Figure 1: An overview of the over-unlearning threat in machine unlearning as a service.
  • Figure 2: An illustration of two types of implications of over-unlearning. The white circle represents the information that the $\bm{\theta}^*$ should unlearn.
  • Figure 3: An example of blending "airplane" sample with "cat" sample (with $\lambda=0.1$).
  • Figure 4: An illustration of over-unlearning using adversarial perturbation. Moving the unlearned data to the decision boundary for unlearning can significantly change the decision boundary of the model.
  • Figure 5: Effectiveness of Pushing-I and Pushing-II for over-unlearning-I when unlearning 10% and 50% training data of a class on CIFAR-10, CIFAR-100, and STL-10.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Definition 1: Over-unlearning