Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

TL;DR

This work reframes LLM unlearning through the linear representation hypothesis, positing that a one-dimensional concept direction in the forget-representation space can be manipulated to not only erase targeted knowledge but also elicit controllable side behaviors and enhanced capabilities. It introduces Representational Addition (RAd) and Representational Ablation (RAb) as two practical interventions and provides theoretical support via linearity results and Levy’s lemma, linking latent and unembedding concept representations. Empirically, RAd and RAb induce measurable changes in truthfulness, sentiment, refusal, and in-context learning across Zephyr-7B and Mistral-7B on WMDP and TruthfulQA benchmarks, with RAd generally enhancing and RAb suppressing the targeted side effects. The findings reveal a double-edged potential: RM can serve as a powerful tool for controllable model behavior and capability amplification, while also presenting risks of unintended leakage through knowledge recovery attacks and robustness concerns. Overall, the paper advances a principled, direction-based approach to machine unlearning with broad implications for safety, alignment, and capability design.

Abstract

We consider representation misdirection (RM), a class of LLM unlearning methods that achieves forgetting by manipulating the forget-representations, that is, latent representations of forget samples. Despite being important, the roles of target vectors used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the linear representation hypothesis. Specifically, if one can somehow identify a one-dimensional representation corresponding to a high-level concept, the linear representation hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning elicits controllable side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models' truth, sentiment, and refusal) and capability enhancement (e.g., improving unlearned models' in-context learning capability). Our findings reveal that this fairly attractive phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing models that require stronger capabilities and controllable behaviors.

Figures (12)

• Figure 1: Layer-wise knowledge recovery attack performance of Logitlens on the WMDP-Biology QA set.
• Figure 2: Finetuning on WMDP-Biology forget-samples (forget), WMDP-Biology retain-samples (forget-relevant), and Wikitext samples (forget-irrelevant). Finetuning on WMDP-Biology forget or forget-relevant samples recovers forgotten knowledge in the WMDP-Cyber domain.
• Figure 3: Context templates for ICL tasks. The zero-shot template is: "Text: [input]\\ nLabel:"
• Figure 4: Prompt template used for sentiment evaluation in Section \ref{['sec:sentiment']}.
• Figure 5: Chat template used for refusal evaluation in Section \ref{['sec:refusal']}.

Theorems & Definitions (11)

• Proposition 3.1
• proof
• Definition 4.1: Unembedding Representation park2024linear
• Theorem 4.2: Theorem 2.2 park2024linear
• proof
• Definition 4.3: Rephrased from Definition 2.3 park2024linear
• Lemma 4.4: Rephrased from Lemma 2.4 park2024linear
• proof
• Lemma 4.5: Lévy's Lemma
• Proposition 4.5