Chain-of-Sanitized-Thoughts: Plugging PII Leakage in CoT of Large Reasoning Models

TL;DR

This work tackles the privacy risk of PII leakage in chain-of-thought reasoning by LRMs and proposes a privacy-first paradigm. It introduces PII-CoT-Bench, a dataset pairing synthetic PII with privacy-aware reasoning traces, and a category-balanced evaluation suite to diagnose leakage across realistic and adversarial scenarios. Through systematic experiments on open-source LRMs, the authors show a capability-dependent effect: strong models benefit most from prompt-based controls, while weaker models require supervised fine-tuning to meaningfully reduce leakage, with minimal utility loss. The study demonstrates that privacy-preserving reasoning can be achieved without sacrificing performance and outlines directions for extending privacy protections through RLHF, architecture-level analyses, and inference-time interventions like activation steering.

Abstract

Large Reasoning Models (LRMs) improve performance, reliability, and interpretability by generating explicit chain-of-thought (CoT) reasoning, but this transparency introduces a serious privacy risk: intermediate reasoning often leaks personally identifiable information (PII) even when final answers are sanitized. We study how to induce privacy-first reasoning, where models reason without exposing sensitive information, using deployable interventions rather than post-hoc redaction. We introduce PII-CoT-Bench, a supervised dataset with privacy-aware CoT annotations, and a category-balanced evaluation benchmark covering realistic and adversarial leakage scenarios. Our results reveal a capability-dependent trend: state-of-the-art models benefit most from prompt-based controls, whereas weaker models require fine-tuning to achieve meaningful leakage reduction. Across models and categories, both approaches substantially reduce PII exposure with minimal degradation in utility, demonstrating that private reasoning can be achieved without sacrificing performance. Overall, we show that private CoT reasoning can be achieved with minimal utility loss, providing practical guidance for building privacy-preserving reasoning systems.

Figures (6)

• Figure 1: Example of PII leakage in GPT-OSS-20B's CoT
• Figure 2: As opposed to existing work where the attack surface shifts to transition phase and redaction model (top), tuned models plug the attack surface and cause models to think privately (bottom)
• Figure 3: Some modified examples from PII-CoT-Bench: The first example shows how the chain of thought refuses to even discuss any PII. In the second example, where age is considered a non-PII, the chain of thought gives out that information without leaking any other PII.
• Figure 4: Dumbbell plot showing baseline performance (black markers) and the effect of supervised fine-tuning (SFT, blue solid lines) and prompt engineering (PE, green dashed lines) across models and metrics. The X-axis represents absolute metric values, with lines indicating improvements or regressions relative to the baseline.
• Figure 5: Privacy-Utility tradeoff across models. Each model-treatment combination is represented by a unique color and marker corresponding to the model family (marker shape). Baseline, supervised fine-tuning (SFT), and prompt engineering (PE) scores are shown for each model, with dashed gray lines connecting the points to indicate the trajectory of changes in privacy and utility.