Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization

Wenhan Wu, Zheyuan Liu, Chongyang Gao, Ren Wang, Kaize Ding

TL;DR

This work addresses the vulnerability of current LLM unlearning methods to relearning attacks by showing that optimizing only the forget loss drives parameters into sharp minima. It introduces StableUN, a bi-level, neighborhood-aware framework that combines forgetting feedback (via perturbations) and remembering feedback (utility preservation) with gradient projection to encourage flatter, more robust parameter regions. The approach is validated on WMDP and MUSE benchmarks, demonstrating improved resistance to relearning and jailbreak attacks while maintaining competitive utility. The results suggest that explicit neighborhood-awareness and gradient harmonization can significantly strengthen the practical guarantees of unlearning in large language models.

Abstract

Current LLM unlearning methods face a critical security vulnerability that undermines their fundamental purpose: while they appear to successfully remove sensitive or harmful knowledge, this ``forgotten" information remains precariously recoverable through relearning attacks. We identify that the root cause is that conventional methods optimizing the forgetting loss at individual data points will drive model parameters toward sharp minima in the loss landscape. In these unstable regions, even minimal parameter perturbations can drastically alter the model's behaviors. Consequently, relearning attacks exploit this vulnerability by using just a few fine-tuning samples to navigate the steep gradients surrounding these unstable regions, thereby rapidly recovering knowledge that was supposedly erased. This exposes a critical robustness gap between apparent unlearning and actual knowledge removal. To address this issue, we propose StableUN, a bi-level feedback-guided optimization framework that explicitly seeks more stable parameter regions via neighborhood-aware optimization. It integrates forgetting feedback, which uses adversarial perturbations to probe parameter neighborhoods, with remembering feedback to preserve model utility, aligning the two objectives through gradient projection. Experiments on WMDP and MUSE benchmarks demonstrate that our method is significantly more robust against both relearning and jailbreaking attacks while maintaining competitive utility performance.

Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization

TL;DR

This work addresses the vulnerability of current LLM unlearning methods to relearning attacks by showing that optimizing only the forget loss drives parameters into sharp minima. It introduces StableUN, a bi-level, neighborhood-aware framework that combines forgetting feedback (via perturbations) and remembering feedback (utility preservation) with gradient projection to encourage flatter, more robust parameter regions. The approach is validated on WMDP and MUSE benchmarks, demonstrating improved resistance to relearning and jailbreak attacks while maintaining competitive utility. The results suggest that explicit neighborhood-awareness and gradient harmonization can significantly strengthen the practical guarantees of unlearning in large language models.

Abstract

Current LLM unlearning methods face a critical security vulnerability that undermines their fundamental purpose: while they appear to successfully remove sensitive or harmful knowledge, this ``forgotten" information remains precariously recoverable through relearning attacks. We identify that the root cause is that conventional methods optimizing the forgetting loss at individual data points will drive model parameters toward sharp minima in the loss landscape. In these unstable regions, even minimal parameter perturbations can drastically alter the model's behaviors. Consequently, relearning attacks exploit this vulnerability by using just a few fine-tuning samples to navigate the steep gradients surrounding these unstable regions, thereby rapidly recovering knowledge that was supposedly erased. This exposes a critical robustness gap between apparent unlearning and actual knowledge removal. To address this issue, we propose StableUN, a bi-level feedback-guided optimization framework that explicitly seeks more stable parameter regions via neighborhood-aware optimization. It integrates forgetting feedback, which uses adversarial perturbations to probe parameter neighborhoods, with remembering feedback to preserve model utility, aligning the two objectives through gradient projection. Experiments on WMDP and MUSE benchmarks demonstrate that our method is significantly more robust against both relearning and jailbreaking attacks while maintaining competitive utility performance.

Paper Structure

This paper contains 36 sections, 13 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Machine Unlearning in Large Language Models
  4. Robustness and adversarial vulnerabilities.
  5. Preliminaries
  6. LLM Unlearning
  7. Relearning Attacks in LLM Unlearning
  8. Methodology
  9. Motivation
  10. Overview
  11. Forgetting Feedback
  12. Remembering Feedback
  13. Feedback aggregation and unified objective.
  14. Experiments
  15. Experiment Setup
  16. ...and 21 more sections

Figures (7)

  • Figure 1: Effects of Unlearning and Relearning on WMDP Bio for Zephyr-7B-beta Model.
  • Figure 2: Visualization of loss landscapes on $D_f$. (a) shows a sharp region from GA, while (b) shows a flatter minimum obtained via our methods. $(\alpha,\beta)$ sampled on a uniform grid. The red arrow indicates the steepest descent direction of the loss surface at (0,0).
  • Figure 3: The bi-level feedback-guided unlearning framework. (a) includes robustness-oriented forgetting feedback with parameter perturbations to simulate relearning attacks and utility-preserving remembering feedback that maintains knowledge through retention evaluation, while (b) shows gradient harmonization, which resolves conflicts between two objectives through orthogonal projection.
  • Figure 5: Evaluations under jailbreak attacks: (a) $\mathcal{MU}$ effectiveness comparison with StableUN added. (b)(c) KL divergence for output token between $f_\text{U}$ and $f_\text{O}$ for GA and NPO, respectively.
  • Figure : (i) Zephyr-7B-beta on WMDP-bio
  • ...and 2 more figures