Sparsity-Aware Unlearning for Large Language Models

TL;DR

This work tackles the privacy risk of memorized data in large language models by exploring machine unlearning under sparsity. It identifies that standard unlearning deteriorates when models are sparsified because pruning restricts updates to a smaller set of surviving weights. To overcome this, it proposes Sparsity-Aware Unlearning (SAU), which couples gradient masking with an importance-aware redistribution grounded in Fisher information to concentrate forgetting updates on critical surviving weights while compensating for pruned capacity. The approach is theoretically justified and empirically validated on TOFU, WDMP, and MUSE benchmarks with Llama models, where SAU consistently improves forgetting performance and preserves utility compared to existing methods, enabling practical privacy-preserving deployment of sparse LLMs.

Abstract

Large Language Models (LLMs) inevitably memorize sensitive information during training, posing significant privacy risks. Machine unlearning has emerged as a promising solution to selectively remove such information without full retraining. However, existing methods are designed for dense models and overlook model sparsification-an essential technique for efficient LLM deployment. We find that unlearning effectiveness degrades substantially on sparse models. Through empirical analysis, we reveal that this degradation occurs because existing unlearning methods require updating all parameters, yet sparsification prunes substantial weights to zero, fundamentally limiting the model's forgetting capacity. To address this challenge, we propose Sparsity-Aware Unlearning (SAU), which decouples unlearning from sparsification objectives through gradient masking that redirects updates to surviving weights, combined with importance-aware redistribution to compensate for pruned parameters. Extensive experiments demonstrate that SAU significantly outperforms existing methods on sparse LLMs, achieving effective forgetting while preserving model utility.

Figures (5)

• Figure 1: Illustration of the unlearning degradation problem under sparsification. Given a query about information in the forget set, a dense Llama-3.1-8B model after unlearning correctly fails to recall the targeted knowledge (producing an incorrect name). However, after applying sparsification to the same unlearned model, it unexpectedly recovers the ability to answer the query accurately, as if the forgotten knowledge has resurfaced. This demonstrates that sparsification can effectively "undo" the unlearning process.
• Figure 2: Aggregate score of different unlearning methods (NPO, SatImp, UNDIAL) on TOFU (forget set 10%) with Llama-3.1-8B under varying sparsity ratios, using three pruning techniques: Magnitude, SparseGPT, and Wanda. All methods exhibit consistent degradation in unlearning effectiveness as sparsity increases, demonstrating the fundamental conflict between unlearning and sparsification.
• Figure 3: The overall pipeline of SAU: (1) calculate influence scores over forget data; (2) generate gradient masks for surviving weights; (3) compute importance-aware redistribution weights; (4) unlearn with masked gradients.
• Figure 4: Ablation studies on (a) top-k ratio and (b) importance-aware redistribution effect on TOFU Forget-10% with Llama-3.1-8B at 50% sparsity.
• Figure 5: Ablation study on importance-aware weight redistribution. (a) Memorization scores: SAU with redistribution achieves lower memorization (better forgetting) across all methods. (b) Utility scores: redistribution substantially improves utility preservation, with an average 32% improvement.

Theorems & Definitions (8)

• Definition 4.1: Forget Set Fisher Information
• Theorem 4.2: Unlearning Effectiveness Bound
• Corollary 4.3: Capacity Loss Under Sparsity
• Proposition 4.4: Fisher Information Compensation
• Remark 1.1
• proof
• proof
• Remark 1.2