Designing to Forget: Deep Semi-parametric Models for Unlearning

Abstract

Recent advances in machine unlearning have focused on developing algorithms to remove specific training samples from a trained model. In contrast, we observe that not all models are equally easy to unlearn. Hence, we introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning. SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters. Empirically, we demonstrate that SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning. Notably, on ImageNet classification, SPMs reduce the prediction gap relative to a retrained (oracle) baseline by $11\%$ and achieve over $10\times$ faster unlearning compared to existing approaches on parametric models. The code is available at https://github.com/amberyzheng/spm_unlearning.

Figures (5)

• Figure 1: Visualization of semi-parametric classifier's boundaries before and after unlearning. Points shown in '${\bm{\mathsfit{X}}}$' are removed from the training set. We visualize: (a) the base semi-parametric model (SPM) trained on the full dataset; (b) the unlearned (oracle) SPM by retraining with samples removed; (c) the unlearned SPM by our test-time deletion. We observe that SPM's unlearned decision boundary by our method in (c) closely matches the one from retraining with the datapoints removed in (b).
• Figure 2: Unlearning with Generative SPM. It consists of three types of layers: (a) Fusion module$g$ connects parametric and non-parametric modules; (b) Non-parametric module$h$ maps a set to another set; (c) Parametric module$f$ maps a vector to a vector. The first row illustrates a standard pipeline for generating a 'bird'. The second row presents unlearning 'bird' via test-time deletion.
• Figure 3: Qualitative results for unlearned SPM. When a class is unlearned (e.g., cats or frogs framed in red), the model avoids generating that concept and produces samples resembling remaining classes (e.g., a truck replacing a cat or a plane replacing a frog). Additionally, the generations from the remaining classes are left unchanged (framed in blue).
• Figure 4: Qualitative comparison of unlearned SPM. When a class is unlearned (e.g., cats or frogs framed in red), the model avoids generating that concept and produces samples resembling remaining classes (e.g., a truck replacing a cat). Additionally, the generations from the remaining classes are left unchanged (framed in blue).
• Figure 5: Generated images of GMM on CIFAR-10. The low-quality outputs demonstrate the limitations of traditional non-parametric models in handling complex generative tasks.

Theorems & Definitions (4)

• Claim 1
• proof
• Claim 1
• proof