Label Smoothing Improves Gradient Ascent in LLM Unlearning

TL;DR

This work addresses the instability of Gradient Ascent (GA) in LLM unlearning by introducing Smoothed Gradient Ascent (SGA), which blends forget data with generated normal data through a tunable smoothing rate $r$ to stabilize updates and preserve model utility. Grounded in Generalized Label Smoothing, SGA reformulates the objective so that the per-step update combines forget and normal data gradients, reducing divergence while maintaining forgetting strength. The authors provide a closed-form expression for an optimal rate $r^{\ast}$ and validate its feasible range theoretically and empirically across TOFU, Harry Potter, and MUSE-NEWS, showing SGA consistently outperforms GA and other baselines on forgetting metrics while mitigating utility loss. The results imply that jointly learning from forget and safe, semantically related data can yield robust unlearning with better safety and performance trade-offs, with practical guidance on selecting $r$ depending on data and model scale. Ethical and reproducibility statements accompany the work, and code/data accessibility is promised to support future research and deployments.

Abstract

LLM unlearning has emerged as a promising approach, aiming to enable models to forget hazardous/undesired knowledge at low cost while preserving as much model utility as possible. Among existing techniques, the most straightforward method is performing Gradient Ascent (GA) w.r.t. the forget data, thereby forcing the model to unlearn the forget dataset. However, GA suffers from severe instability, as it drives updates in a divergent direction, often resulting in drastically degraded model utility. To address this issue, we propose Smoothed Gradient Ascent (SGA). SGA combines the forget data with multiple constructed normal data through a tunable smoothing rate. Intuitively, this extends GA from learning solely on the forget data to jointly learning across both forget and normal data, enabling more stable unlearning while better preserving model utility. Theoretically, we provide the theoretical guidance on the selection of the optimal smoothing rate. Empirically, we evaluate SGA on three benchmarks: TOFU, Harry Potter, and MUSE-NEWS. Experimental results demonstrate that SGA consistently outperforms the original Gradient Ascent (GA) method across all metrics and achieves top-2 performance among all baseline methods on several key metrics.

Figures (3)

• Figure 1: SGA pipeline. SGA extends Gradient Ascent (SGA, $r=0$) by incorporating normal data generated from the normal model, which is combined with the original forget data to form a distribution over $K$ labels. The smoothing rate $r$ regulates the balance between learning and forgetting, and varying its value leads to different outcomes. Through this mechanism, SGA effectively mitigates the divergence issue of GA, which arises from maximizing the loss solely on the forget set.
• Figure 2: Following Section \ref{['section_optimal_r']}, we estimate the sign of $r^{\star}$ for each forget data instance in the TOFU benchmark forget01 dataset, which corresponds to the sign of $\langle g_f, u \rangle$ in Equation \ref{['eqn:rstar']}.
• Figure 3: Different smoothing rates yield distinct output probabilities for critical tokens across base LLMs. As shown in the figure, the forget data highlights Basil Mahfouz Al-Kuwaiti’s achievements in French literary style. In the LLaMA2-7B model, training with $r=-0.8$ assigns a much higher probability to the token "French," whereas training with $r=0.2$ makes the model far less likely to produce this privacy-sensitive token.