LMEraser: Large Model Unlearning through Adaptive Prompt Tuning

TL;DR

LMEraser addresses the privacy-driven need to erase data from large models by separating public and private data and freezing a backbone trained on public data. It introduces adaptive prompt tuning with HierCluster-based private data clustering to create cluster-specific prompts and classifier heads, enabling exact unlearning by retraining only the affected prompts/heads. Empirical results show a substantial reduction in unlearning costs (around 100x) while maintaining competitive accuracy across multiple datasets and backbones, demonstrating scalability to large models. The approach offers practical impact for complying with privacy rights like the right to be forgotten without full model retraining.

Abstract

To address the growing demand for privacy protection in machine learning, we propose a novel and efficient machine unlearning approach for \textbf{L}arge \textbf{M}odels, called \textbf{LM}Eraser. Existing unlearning research suffers from entangled training data and complex model architectures, incurring extremely high computational costs for large models. LMEraser takes a divide-and-conquer strategy with a prompt tuning architecture to isolate data influence. The training dataset is partitioned into public and private datasets. Public data are used to train the backbone of the model. Private data are adaptively clustered based on their diversity, and each cluster is used to optimize a prompt separately. This adaptive prompt tuning mechanism reduces unlearning costs and maintains model performance. Experiments demonstrate that LMEraser achieves a $100$-fold reduction in unlearning costs without compromising accuracy compared to prior work. Our code is available at: \url{https://github.com/lmeraser/lmeraser}.

Figures (10)

• Figure 1: Comparing sharding-based unlearning and LMEraser. Sharding-based unlearning methods require retraining the affected shard model (around $85$M parameters), while LMEraser efficiently retrains the affected prompt (around $12$k parameters), significantly reducing retraining costs.
• Figure 2: Overview model training process of LMEraser. (1) Partitioning training data into public and private datasets and pre-training backbone on public data. (2) Clustering private data based on diversity. (3) Tuning prompts and classifier heads for each cluster.
• Figure 3: Comparative analysis of test accuracy over epochs: LMEraser vs. baseline methods across various datasets with ViT-B-22K.
• Figure 4: Performance evolution of LMEraser with Swin-B-22K: Test accuracy and average accuracy trends across epochs in 8 GPUs.
• Figure 5: LMEraser's test accuracy with various unlearning privacy data ratio and number of prompts (with ViT-B-22K backbone on CIFAR100 private dataset).

Theorems & Definitions (1)

• Definition 1: Pixel-frame Visual Prompt