Layered Unlearning for Adversarial Relearning

TL;DR

The paper investigates why post-training updates to large language models are brittle and prone to being bypassed by adversarial relearning. It introduces Layered Unlearning (LU), a k-fold sequential forgetting framework that creates multiple, context-dependent inhibitors by progressively forgetting data folds while retaining others, thereby reducing the chances that relearning can recover the full forgotten information. Through synthetic tasks and extensive LLM experiments (WMDP, MMLU, Years), LU demonstrates improved robustness to adversarial relearning and reveals a gap between MCQ-based and corpus-based attacks, highlighting the limits of current unlearning methods. The findings suggest that robust post-training updates may require layered, interpretable inhibitors and point toward more modular, controllable strategies for alignment and safety in LLMs.

Abstract

Our goal is to understand how post-training methods, such as fine-tuning, alignment, and unlearning, modify language model behavior and representations. We are particularly interested in the brittle nature of these modifications that makes them easy to bypass through prompt engineering or relearning. Recent results suggest that post-training induces shallow context-dependent ``circuits'' that suppress specific response patterns. This could be one explanation for the brittleness of post-training. To test this hypothesis, we design an unlearning algorithm, Layered Unlearning (LU), that creates distinct inhibitory mechanisms for a growing subset of the data. By unlearning the first $i$ folds while retaining the remaining $k - i$ at the $i$th of $k$ stages, LU limits the ability of relearning on a subset of data to recover the full dataset. We evaluate LU through a combination of synthetic and large language model (LLM) experiments. We find that LU improves robustness to adversarial relearning for several different unlearning methods. Our results contribute to the state-of-the-art of machine unlearning and provide insight into the effect of post-training updates.

Figures (8)

• Figure 1: Left: An illustration of LU with social security numbers (SSNs). The SSNs are partitioned into disjoint sets $(A, B, C)$. Top: Standard unlearning minimizes performance on $A\cup B \cup C$ while retaining general capabilities on a retain set $R$ (e.g., MMLU). Bottom: In LU, we sequentially unlearn the sequence $\{A, A\cup B, A\cup B \cup C\}$ while retaining the sequence $\{B \cup C \cup R, C \cup R, R\}$. Middle: As a result, relearning $B$ improves performance on $C$ but not $A$. In contrast, training on any subset improves performance across the board for standard methods. Right: We hypothesize that unlearning the full set introduces a context-dependent shared inhibitor$I_{ABC}$ that suppresses the information and that subsequent relearning removes $I_{ABC}$. The structure of LU is designed to create several distinct inhibitors $I_{A}, I_{AB}, I_{ABC}$ that cover different folds of the data. Relearning on $B$ removes $I_{AB}$ and $I_{ABC}$, but leaves $I_{A}$ active.
• Figure 2: A depiction of Layered Unlearning in our 2D logistic regression example. Scatter plots represent the data, which consists of a uniform distribution (Class 0) and a mixture of Gaussians (Class 1) that is split into three subsets, $A$, $B$, and $R$. The goal of unlearning is to forget $A, B$ while retaining $R$. Our classifiers are trained with logistic regression with radial basis functions spaced out in a grid. We show the weights as a heatmap on the grid. The top left shows $\theta_0$, the initial trained model. Across the top row, we illustrate the effect of joint unlearning $\theta_1 = (\theta_0, A \cup B, R)$ and subsequent relearning on $B$. Note that relearning $B$ also relearns $A$. The corresponding classification logits along $Y=0$ are shown below. Notice how learning on $B$ generalizes to the area around $(0, 0)$. In the bottom row, we show the steps of Layered Unlearning. First, we compute $\theta'_0 = U(\theta_0, A, R\cup B),$ shown in the bottom left, then we compute $\theta_1' = U(\theta_0', A \cup B, R),$ shown in the bottom middle. The logit plot shows the clear effect on the logits near $(0, 0)$. The bottom right shows the effect of subsequently relearning $B$, while performance on $B$ still improves, it no longer generalizes to $A$.
• Figure 3: Model accuracy after relearning different folds of the data for an experiment with Layered RMU (L-RMU) and the folds $\{A, B, C\}$ in order. Each row shows the performance per fold for different relearning subsets. Notice that values below the diagonal are lower than values above the diagonal. This shows that L-RMU introduces a one-way barrier to relearning: relearning on $B$ regains performance on $C$ but not on $A$.
• Figure 4: The performance trajectory of LU on two folds $A, B$. Normally, unlearning methods lose performance on $A, B$ jointly and directly head towards the red point. However, we propose performing LU to retain performance on $B$ while forgetting $A$ and then forgetting both folds.
• Figure 5: We show the accuracy progression of forgetting three sets $A, B, C$ in that order using RMU on WMDP. The vertical dotted gray lines show when we move to forgetting the next fold. Note that the $A$ accuracy drops in the first iteration of forgetting and remains low. The accuracy of $B$ remains high until the second iteration of forgetting, and then drops and remains low. Finally, the accuracy of $C$ remains high until the third half of forgetting, when it drops.