Selective Forgetting for Large Reasoning Models

Abstract

Large Reasoning Models (LRMs) generate structured chains of thought (CoTs) before producing final answers, making them especially vulnerable to knowledge leakage through intermediate reasoning steps. Yet, the memorization of sensitive information in the training data such as copyrighted and private content has led to ethical and legal concerns. To address these issues, selective forgetting (also known as machine unlearning) has emerged as a potential remedy for LRMs. However, existing unlearning methods primarily target final answers and may degrade the overall reasoning ability of LRMs after forgetting. Additionally, directly applying unlearning on the entire CoTs could degrade the general reasoning capabilities. The key challenge for LRM unlearning lies in achieving precise unlearning of targeted knowledge while preserving the integrity of general reasoning capabilities. To bridge this gap, we in this paper propose a novel LRM unlearning framework that selectively removes sensitive reasoning components while preserving general reasoning capabilities. Our approach leverages multiple LLMs with retrieval-augmented generation (RAG) to analyze CoT traces, identify forget-relevant segments, and replace them with benign placeholders that maintain logical structure. We also introduce a new feature replacement unlearning loss for LRMs, which can simultaneously suppress the probability of generating forgotten content while reinforcing structurally valid replacements. Extensive experiments on both synthetic and medical datasets verify the desired properties of our proposed method.

Figures (6)

• Figure 1: Comparison of unlearning performance on the forget data of R-TOFU.
• Figure 2: Comparison of retain utility on the retain data of R-TOFU.
• Figure 3: Comparison of unlearning performance on the forget data of the adopted medical-o1-reasoning dataset.
• Figure 4: Comparison of retain utility on the retain data of medical-o1-reasoning.
• Figure 5: Performance comparison of Reasoning-Llama-3.2-1B on the R-TOFU benchmark before and after unlearning on a 5$\%$ unlearning ratio. (a) Baseline model fine-tuned on the full R-TOFU training set; (b) Model unlearned using the R$^2$MU method; (c) Model unlearned using the proposed FRUL loss.