CoUn: Empowering Machine Unlearning via Contrastive Learning

TL;DR

CoUn proposes a contrastive-learning–driven MU framework that applies self-supervised learning to retain data to indirectly reposition forget-data representations toward semantically similar retain clusters, while using supervised learning to preserve retain-data cluster integrity. This combination promotes higher forget-quality (through misclassification of forget data) and utility (through retained accuracy) compared to prior MU methods, and its CL module can enhance existing MU baselines when integrated. The approach is evaluated across CIFAR-10/100 and TinyImageNet with multiple architectures, showing consistent gains in unlearning effectiveness, even under sequential and varying forget scenarios. Theoretical analysis supports the observed behavior by bounding forget-data misclassification and showing retention of utility, highlighting CoUn’s practical impact for privacy-preserving model maintenance.

Abstract

Machine unlearning (MU) aims to remove the influence of specific "forget" data from a trained model while preserving its knowledge of the remaining "retain" data. Existing MU methods based on label manipulation or model weight perturbations often achieve limited unlearning effectiveness. To address this, we introduce CoUn, a novel MU framework inspired by the observation that a model retrained from scratch using only retain data classifies forget data based on their semantic similarity to the retain data. CoUn emulates this behavior by adjusting learned data representations through contrastive learning (CL) and supervised learning, applied exclusively to retain data. Specifically, CoUn (1) leverages semantic similarity between data samples to indirectly adjust forget representations using CL, and (2) maintains retain representations within their respective clusters through supervised learning. Extensive experiments across various datasets and model architectures show that CoUn consistently outperforms state-of-the-art MU baselines in unlearning effectiveness. Additionally, integrating our CL module into existing baselines empowers their unlearning effectiveness.

Figures (12)

• Figure 1: Representation space of the Retrain model trained with ResNet-18 and CIFAR-10, excluding 'truck' class samples (left) and excluding 10% randomly selected samples (right). Small dots represent retain samples from different clusters, while larger dots indicate forget samples classified into clusters of retain samples that exhibit the highest semantic similarity to them.
• Figure 2: CoUn framework. Two augmented views are generated from a batch of retain image samples $\bm{I}$. These views are processed by the feature extractor $f_{\bm{\theta}_u}$, yielding retain representations ($\bm{Z}$, $\bm{Z}^{\prime}$). A CL module adjusts the representations, while supervised learning applied via the classifier head $h_{\bm{\theta}_u}$ enforces their cluster separation.
• Figure 3: Representation space of FT and CoUn unlearned models (rows). Columns correspond to two forgetting scenarios: class-wise ('truck') and random (10% forget ratio). The Original model is trained on CIFAR-10 using ResNet-18. Small dots represent retain samples from different clusters, while larger dots indicate forget samples classified into the corresponding clusters. To achieve effective unlearning, CoUn adjusts representations based on semantic similarity, pushing forget representations into clusters of other retain samples with the highest semantic similarity to them, and preserving retain representations within their clusters.
• Figure 4: Percentage improvement from integrating CoUn's CL module into baseline methods. Incorporating our CL module consistently improves baseline unlearning performance compared to the original MU methods (without CL). The performance improvements further increase with a 50% forget ratio.
• Figure 5: Performance comparison of MU methods on CIFAR-100 with ResNet-18, where 10% (left) and 50% (right) of training data are randomly selected as forget data. The best performance of each method is reported. CoUn outperforms all baselines, and integrating its CL module empowers baseline performance. Although CL increases computational cost, the performance improvement persists even with the same computational budget.

Theorems & Definitions (4)

• Theorem 1
• proof
• Lemma 1
• proof