Unsupervised decoding of encoded reasoning using language model interpretability

TL;DR

The paper addresses whether current mechanistic interpretability methods can reveal encoded or non-human-readable reasoning in large language models. It constructs a controlled testbed by finetuning a reasoning model to ROT-13 encoded chain-of-thought while preserving English outputs, and develops an unsupervised decoding pipeline that combines logit lens with automated paraphrasing. The results show that logit lens can align intermediate-to-late layer representations with English translations of encoded thinking, and the unsupervised decoding pipeline can recover substantial portions of the original reasoning transcripts (about 72% correctness on a 50-prompt test), with peak performance around layer 58 and improvements from thresholding. This work provides an initial framework for evaluating interpretability methods against encoded reasoning and informs oversight of increasingly capable AI systems.

Abstract

As large language models become increasingly capable, there is growing concern that they may develop reasoning processes that are encoded or hidden from human oversight. To investigate whether current interpretability techniques can penetrate such encoded reasoning, we construct a controlled testbed by fine-tuning a reasoning model (DeepSeek-R1-Distill-Llama-70B) to perform chain-of-thought reasoning in ROT-13 encryption while maintaining intelligible English outputs. We evaluate mechanistic interpretability methods--in particular, logit lens analysis--on their ability to decode the model's hidden reasoning process using only internal activations. We show that logit lens can effectively translate encoded reasoning, with accuracy peaking in intermediate-to-late layers. Finally, we develop a fully unsupervised decoding pipeline that combines logit lens with automated paraphrasing, achieving substantial accuracy in reconstructing complete reasoning transcripts from internal model representations. These findings suggest that current mechanistic interpretability techniques may be more robust to simple forms of encoded reasoning than previously understood. Our work provides an initial framework for evaluating interpretability methods against models that reason in non-human-readable formats, contributing to the broader challenge of maintaining oversight over increasingly capable AI systems.

Figures (7)

• Figure 1: Supervised finetuning a reasoning model to use encoded chain-of-thought.
• Figure 2: Logit lens analysis of model activations during encoded reasoning. A. Most likely token identified by logit lens across four model layers (20, 40, 60, 70). The actual model output is provided, as is an unencrypted translation (green shading). Token cells are colored by their probability. B. Probability that the token identified by logit lens is the English translation of the ROT-13 word the model is completing, as a function of model layer. Averaged over the full thinking transcript over $50$ prompts, with 95% confidence intervals.
• Figure 3: Unsupervised translation pipeline based on logit lens and automated paraphrasing.
• Figure 4: Grader scores across translation methods.
• Figure 5: Supervised finetuning dataset and training run.