Machine Unlearning

TL;DR

The paper tackles the challenge of enabling efficient, provable unlearning in ML models to satisfy rights to erasure. It introduces SISA training—sharding data, isolating shard models, slicing data within shards, and aggregating outputs—to dramatically reduce retraining time when removing a user’s data, while preserving accuracy on simple tasks and leveraging transfer learning for complex tasks. Through analytical time models and extensive experiments on MNIST, SVHN, Purchase, and Imagenet-family datasets, it demonstrates substantial speed-ups under realistic unlearning loads, and discusses regimes where benefits are most pronounced. It also extends the framework to distribution-aware sharding to handle nonuniform unlearning requests, highlighting practical considerations for deployment, verification, and governance.

Abstract

Once users have shared their data online, it is generally difficult for them to revoke access and ask for the data to be deleted. Machine learning (ML) exacerbates this problem because any model trained with said data may have memorized it, putting users at risk of a successful privacy attack exposing their information. Yet, having models unlearn is notoriously difficult. We introduce SISA training, a framework that expedites the unlearning process by strategically limiting the influence of a data point in the training procedure. While our framework is applicable to any learning algorithm, it is designed to achieve the largest improvements for stateful algorithms like stochastic gradient descent for deep neural networks. SISA training reduces the computational overhead associated with unlearning, even in the worst-case setting where unlearning requests are made uniformly across the training set. In some cases, the service provider may have a prior on the distribution of unlearning requests that will be issued by users. We may take this prior into account to partition and order data accordingly, and further decrease overhead from unlearning. Our evaluation spans several datasets from different domains, with corresponding motivations for unlearning. Under no distributional assumptions, for simple learning tasks, we observe that SISA training improves time to unlearn points from the Purchase dataset by 4.63x, and 2.45x for the SVHN dataset, over retraining from scratch. SISA training also provides a speed-up of 1.36x in retraining for complex learning tasks such as ImageNet classification; aided by transfer learning, this results in a small degradation in accuracy. Our work contributes to practical data governance in machine unlearning.

Figures (15)

• Figure 1: Unlearning (red arrow) is hard because there exists no function that measures the influence of augmenting the dataset $\mathcal{D}$ with point $d_u$ and fine-tuning a model $M_A$ already trained on $\mathcal{D}$ to train (left blue arrow) a model $M_B$ for $\mathcal{D}+\{d_u\}$. This makes it impossible to revert to model $M_A$ without saving its parameter state before learning about $d_u$. We call this model slicing (short green arrow). In the absence of slicing, one must retrain (curved green arrow) the model without $d_u$, resulting in a model $M_C$ that is different from the original model $M_A$.
• Figure 2: SISA training: data is divided in shards, which are themselves divided into slices. One constituent model is trained on each shard by presenting it with incrementally many slices and saving its parameters before the training set is augmented with a new slice. When data needs to be unlearned, only one of the constituent models whose shards contains the point to be unlearned needs to be retrained --- retraining can start from the last parameter values saved before including the slice containing the data point to be unlearned.
• Figure 3: We validate the linear relationship (within error) between training time and the number of samples trained on. Measurements are obtained on increments of 10% of the dataset size. We repeat 5 times to report mean and variance, on SVHN and Purchase.
• Figure 4: We compare the experimental accuracy of SISA training (with different number of shards) with the two baselines on three datasets: SVHN, Purchase, and MNIST. It is clear that SISA training provides higher accuracy than the $\frac{1}{S}$ fraction baseline, along with less retraining time than the batch $K$ baseline especially when the number of unlearning request is small.
• Figure 5: Performance of single model trained with data slicing. We train each model 5 times for each number of slices on the SVHN and Purchase datasets, respectively, and plot the history of validation accuracy and confidence intervals against the number of training epochs. For a small number of epochs, models with more slicing have lower accuracy, due to the fact that they have significantly less amount of data at the beginning. As the number of epochs grows and the accuracy reaches a plateau, the accuracy of models converges.

Theorems & Definitions (1)

• Definition 3.1