The Unseen Threat: Residual Knowledge in Machine Unlearning under Perturbed Samples

TL;DR

This work reveals a covert privacy risk in machine unlearning: even when unlearning methods are certified to be indistinguishable from a retrained model on original forget data, subtle adversarial perturbations around forget samples can cause persistent, residual knowledge in the unlearned model. It formalizes this risk with the residual-knowledge ratio $r_\tau$, and proves that perturbation-based disagreement is inevitable in high dimensions, motivating a robustness-aware mitigation. To address this, the authors propose RURK, a fine-tuning objective that penalizes the unlearned model's ability to re-recognize perturbed forget samples while preserving performance on retained data; they provide practical algorithmic details and report results across CIFAR-5, CIFAR-10, and ImageNet-100 showing reduced residual knowledge and competitive unlearning utility. The findings highlight a fundamental trade-off between indistinguishability guarantees and robustness to local perturbations, with implications for privacy protection and the design of certification frameworks in real-world unlearning systems.

Abstract

Machine unlearning offers a practical alternative to avoid full model re-training by approximately removing the influence of specific user data. While existing methods certify unlearning via statistical indistinguishability from re-trained models, these guarantees do not naturally extend to model outputs when inputs are adversarially perturbed. In particular, slight perturbations of forget samples may still be correctly recognized by the unlearned model - even when a re-trained model fails to do so - revealing a novel privacy risk: information about the forget samples may persist in their local neighborhood. In this work, we formalize this vulnerability as residual knowledge and show that it is inevitable in high-dimensional settings. To mitigate this risk, we propose a fine-tuning strategy, named RURK, that penalizes the model's ability to re-recognize perturbed forget samples. Experiments on vision benchmarks with deep neural networks demonstrate that residual knowledge is prevalent across existing unlearning methods and that our approach effectively prevents residual knowledge.

Figures (13)

• Figure 1: The re-trained (brown) and unlearned (green) models are statistically similar but may have slightly different decision boundaries (left), leading to disagreements on forget samples (right). Checkmarks and crosses on the images indicate correct and incorrect predictions from the re-trained model (top) and the unlearned model (bottom), respectively. Ideally---as shown with forget sample 1---both models should behave consistently across the original and all perturbed inputs. Residual knowledge in machine unlearning is illustrated by comparing prediction correctness: for forget sample 2, both models agree on the original sample, but the unlearned model correctly predicts more of its perturbed variants. see Appendix \ref{['app:intro-figure-setting']} for experimental details.
• Figure 2: Residual knowledge $\hat{r}_\tau({\mathcal{S}}_f)$ of the proposed RURK, Original, and other unlearning methods across two unlearning scenarios, evaluated under varying perturbation norms $\tau$.
• Figure 3: Performance summary and residual knowledge (following Table \ref{['tab:acc']} and Figure \ref{['fig:RK-ratio']}) of selected unlearning methods on ImageNet-100.
• Figure C.4: Residual knowledge $\hat{r}_\tau({\mathcal{S}}_f)$ of the proposed RURK, Original, and other unlearning methods on small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms $\tau$, using Gaussian noise ($p=2$) to draw $c=100$ samples from ${\mathcal{B}}_p({\mathbf{x}}, \tau)$.
• Figure C.5: Residual knowledge $\hat{r}_\tau({\mathcal{S}}_f)$ of the proposed RURK, Original, and other unlearning methods on small CIFAR-5 with sample unlearning, evaluated under varying perturbation norms $\tau$, using targeted FGSM ($p=\infty$) to draw $c=100$ samples from ${\mathcal{B}}_p({\mathbf{x}}, \tau)$.

Theorems & Definitions (10)

• Definition 1: $(\epsilon, \delta)$-indistinguishability
• Proposition 1: Adversarial example on a model is less indistinguishable
• Proposition 2: Inevitable disagreement
• Lemma A.1: Probabilistic indistinguishability
• proof
• Lemma A.2: milman1986asymptotic
• Lemma A.3: Lower bound on disagreement probability
• proof
• Lemma A.4: Bounding adversarial disagreement with the residual knowledge.
• proof