UnlearnShield: Shielding Forgotten Privacy against Unlearning Inversion

TL;DR

Unlearning inversion threatens privacy by leaking information through the cosine-space directions of the unlearning parameter difference $Δ = θ_u - θ_o$. The authors propose UnlearnShield, a post-processing defense that perturbs to form $Δ^* = Δ + δ$, optimizing a composite loss $L_{total} = L_{privacy} + λ_1 L_{acc} + λ_2 L_{forget}$ to maximize cosine distance between $Δ$ and $Δ^*$ while constraining perturbation magnitude and preserving forgetting via a forgetting-consistency mechanism. The approach incorporates an AIM-based perturbation initialization and a forgetting-oriented loss, achieving a favorable privacy-utility trade-off on CIFAR10 and STL10 with ResNet18 under single-point unlearning. Experimental results show higher privacy metrics (LPIPS, lower SSIM) and minimal degradation in accuracy and forgetting compared to baselines, indicating practical effectiveness. The work delivers the first dedicated defense against unlearning inversion and demonstrates robustness across variants, with potential for extension beyond the computer-vision domain.

Abstract

Machine unlearning is an emerging technique that aims to remove the influence of specific data from trained models, thereby enhancing privacy protection. However, recent research has uncovered critical privacy vulnerabilities, showing that adversaries can exploit unlearning inversion to reconstruct data that was intended to be erased. Despite the severity of this threat, dedicated defenses remain lacking. To address this gap, we propose UnlearnShield, the first defense specifically tailored to counter unlearning inversion. UnlearnShield introduces directional perturbations in the cosine representation space and regulates them through a constraint module to jointly preserve model accuracy and forgetting efficacy, thereby reducing inversion risk while maintaining utility. Experiments demonstrate that it achieves a good trade-off among privacy protection, accuracy, and forgetting.

Figures (5)

• Figure 1: Post-processing defense pipeline against UIA.
• Figure 2: Visualization of defense results of various methods against UIA recon1, with CIFAR10 cifar and ResNet18 resnet18.
• Figure 3: Visualization results under UIA.
• Figure 4: Evaluation of the impact of different $\lambda_1$ and $\lambda_2$ on privacy and usability. Privacy is measured by SSIM ($\uparrow$) and LPIPS ($\downarrow$), while usability is measured by ACC ($\uparrow$) and OutDiff ($\downarrow$).
• Figure 5: Visualization results under adaptive UIA.