On Effects of Steering Latent Representation for Large Language Model Unlearning

TL;DR

This work analyzes Representation Misdirection for Unlearning (RMU), a method that steers forget-sample representations away from the model while preserving retain-sample representations. The authors provide a theoretical link between RMU and reduced token confidence, and show that the effectiveness depends on the steering coefficient and the unlearned layer; to address layer dependence, they introduce Adaptive RMU, which adjusts the forgetting strength adaptively by layer. Empirically, Adaptive RMU outperforms prior methods on forget tasks and maintains overall utility, while also enhancing robustness to adversarial jailbreak attacks. The results offer practical guidance for designing unlearning mechanisms in large language models and highlight a path toward safer LLM deployment with low computational overhead.

Abstract

Representation Misdirection for Unlearning (RMU), which steers model representation in the intermediate layer to a target random representation, is an effective method for large language model (LLM) unlearning. Despite its high performance, the underlying cause and explanation remain underexplored. In this paper, we theoretically demonstrate that steering forget representations in the intermediate layer reduces token confidence, causing LLMs to generate wrong or nonsense responses. We investigate how the coefficient influences the alignment of forget-sample representations with the random direction and hint at the optimal coefficient values for effective unlearning across different network layers. We show that RMU unlearned models are robust against adversarial jailbreak attacks. Furthermore, our empirical analysis shows that RMU is less effective when applied to the middle and later layers in LLMs. To resolve this drawback, we propose Adaptive RMU--a simple yet effective alternative method that makes unlearning effective with most layers. Extensive experiments demonstrate that Adaptive RMU significantly improves the unlearning performance compared to prior art while incurring no additional computational cost.

Figures (6)

• Figure 1: Noise sensitivity of layer $g^{(l:k)}$, for $k \in [3...31]$ in base Zephyr-7B, base Llama-3-8B, base Mistral-7B, and RMU Zephyr-7B model. In the base models, a deeper layer has lower noise sensitivity, while the noise sensitivity is minimized in the RMU model (compress noise into $h^{(7)}$, the noise sensitivity of layer $k=8$ is minimized).
• Figure 2: The distribution of MaxLogit (a-d) on WMDP Q&A sets with different coefficient $c$ of the base Zephyr-7B and RMU Zephyr-7B models ($l=7$). The distribution of $\cos(\bm u, h^{(l)})$ (e-h) of the RMU Zephyr-7B model ($l=7$).
• Figure 3: Average accuracy of WMDP (Biology and Cyber) (left) and MMLU with different coefficient $c$ (right).
• Figure 4: $\ell^2$-norm of forget-sample representation.
• Figure 5: Q&A accuracy of RMU and Adaptive RMU Zephyr-7B models on WMDP-Biology, WMDP-Cyber, and MMLU w.r.t unlearn layer $l$ from the third to the last layer.

Theorems & Definitions (3)

• Definition 1
• Proposition 0
• proof