Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Loss-Free Machine Unlearning

Jack Foster, Stefan Schoepf, Alexandra Brintrup

TL;DR

The paper tackles the challenge of unlearning in neural networks under data-forgetting scenarios without retraining or labeled data. It introduces Loss-Free Selective Synaptic Dampening (LFSSD), which substitutes the Fisher information diagonal with a gradient-based sensitivity ${\Omega_i}$ computed from the squared output norm, enabling forgetting using only unlabelled forget samples $\mathcal{D}_f$. Empirical evaluation on ResNet-18 and Vision Transformers across CIFAR-100, CIFAR-20, and CIFAR-10 demonstrates that LFSSD achieves memory-efficient, competitive forgetting performance, comparable to or better than SSD and other retraining-free baselines. The method reduces computational and storage burdens, offering practical applicability in regulated settings, though it depends on hyperparameters $\alpha$ and $\lambda$ and assumes reliable forget-set signals. Overall, LFSSD broadens accessible unlearning by removing the need for labelled data or full dataset retention, while maintaining robust performance on the retained data.

Abstract

We present a machine unlearning approach that is both retraining- and label-free. Most existing machine unlearning approaches require a model to be fine-tuned to remove information while preserving performance. This is computationally expensive and necessitates the storage of the whole dataset for the lifetime of the model. Retraining-free approaches often utilise Fisher information, which is derived from the loss and requires labelled data which may not be available. Thus, we present an extension to the Selective Synaptic Dampening algorithm, substituting the diagonal of the Fisher information matrix for the gradient of the l2 norm of the model output to approximate sensitivity. We evaluate our method in a range of experiments using ResNet18 and Vision Transformer. Results show our label-free method is competitive with existing state-of-the-art approaches.

Loss-Free Machine Unlearning

TL;DR

The paper tackles the challenge of unlearning in neural networks under data-forgetting scenarios without retraining or labeled data. It introduces Loss-Free Selective Synaptic Dampening (LFSSD), which substitutes the Fisher information diagonal with a gradient-based sensitivity computed from the squared output norm, enabling forgetting using only unlabelled forget samples . Empirical evaluation on ResNet-18 and Vision Transformers across CIFAR-100, CIFAR-20, and CIFAR-10 demonstrates that LFSSD achieves memory-efficient, competitive forgetting performance, comparable to or better than SSD and other retraining-free baselines. The method reduces computational and storage burdens, offering practical applicability in regulated settings, though it depends on hyperparameters and and assumes reliable forget-set signals. Overall, LFSSD broadens accessible unlearning by removing the need for labelled data or full dataset retention, while maintaining robust performance on the retained data.

Abstract

We present a machine unlearning approach that is both retraining- and label-free. Most existing machine unlearning approaches require a model to be fine-tuned to remove information while preserving performance. This is computationally expensive and necessitates the storage of the whole dataset for the lifetime of the model. Retraining-free approaches often utilise Fisher information, which is derived from the loss and requires labelled data which may not be available. Thus, we present an extension to the Selective Synaptic Dampening algorithm, substituting the diagonal of the Fisher information matrix for the gradient of the l2 norm of the model output to approximate sensitivity. We evaluate our method in a range of experiments using ResNet18 and Vision Transformer. Results show our label-free method is competitive with existing state-of-the-art approaches.
Paper Structure (4 sections, 3 equations, 1 figure, 7 tables)

This paper contains 4 sections, 3 equations, 1 figure, 7 tables.

Table of Contents

  1. Introduction & Related Work
  2. Methods & Results
  3. Conclusion
  4. Appendix

Figures (1)

  • Figure 1: Loss-free parameter importance estimation as a replacement for the label-dependent parameter importance estimation of SSD resulting in the LFSSD method.