Evaluating the Defense Potential of Machine Unlearning against Membership Inference Attacks

TL;DR

The paper investigates whether machine unlearning (MU) can act as an active defense against membership inference attacks (MIAs) after training. It systematically compares three MU methods—NegGrad, SCRUB, and SFTC—across four datasets (CIFAR-10, MuFac, Purchase-100, Texas-100) and three model complexities, evaluating privacy-utility trade-offs at each unlearning epoch. The study finds that MU can reduce the forget-set MIA signal, but effects are highly algorithm- and data-dependent, with a notable divergence effect seen in SFTC where forgetting weakens the forget signal while strengthening the retain signal; SCRUB generally offers more balanced results, and NegGrad often yields broader degradation. It also highlights critical practical considerations, such as pronounced learning-rate sensitivity and the risk of entailing predictive collapse with long unlearning, underscoring that MU is not a universal privacy safeguard and should be combined with careful auditing and possibly other privacy tools.

Abstract

Membership Inference Attacks (MIAs) pose a significant privacy risk by enabling adversaries to determine if a specific data point was part of a model's training set. This work empirically investigates whether MU algorithms can function as a targeted, active defense mechanism, in scenarios where a privacy audit identifies specific classes or individuals as highly susceptible to MIAs post-training. By 'dulling' the model's categorical memory of these samples, the process effectively mitigates the membership signal and reduces the MIA success rate for the most vulnerable users. We evaluate the defense potential of three MU algorithms, Negative Gradient (neg grad), SCalable Remembering and Unlearning unBound (SCRUB), and Selective Fine-tuning and Targeted Confusion (SFTC), across four diverse datasets and three complexity-based model groups. Our findings reveal that MU can function as a countermeasure against MIAs, though its success is critically contingent on algorithm choice, model capacity, and a profound sensitivity to learning rates. While Negative Gradient often induces a generalized degradation of membership signals across both forget and retain set, SFTC identifies a critical ``divergence effect'' where targeted forgetting reinforces the membership signal of retained data. Conversely, SCRUB provides a more balanced defense with minimal collateral impact on MIA perspective.

Figures (12)

• Figure 1: MIA - Flow Diagram
• Figure 2: Machine Unlearning - Flow Diagram
• Figure 3: These graphs illustrate the Membership Inference Attack (MIA) performance at each epoch of the unlearning process. This specific set of results corresponds to the first group of experiments, utilizing the Negative Gradient unlearning algorithm.
• Figure 4: These graphs illustrate the Membership Inference Attack (MIA) performance at each epoch of the unlearning process. This specific set of results corresponds to the second group of experiments, utilizing the Negative Gradient unlearning algorithm.
• Figure 5: These graphs illustrate the Membership Inference Attack (MIA) performance at each epoch of the unlearning process. This specific set of results corresponds to the third group of experiments, utilizing the Negative Gradient unlearning algorithm.