Rank-1 LoRAs Encode Interpretable Reasoning Signals

TL;DR

This work investigates how reasoning capabilities in large language models can arise from minimal parameter changes by training a rank-1 LoRA to adapt all projection matrices in Qwen-2.5-32B-Instruct on a reasoning-focused dataset. The authors show the LoRA recovers 73-90% of the performance gain of full finetuning while using only a tiny fraction of trainable parameters, and they provide interpretable evidence that LoRA directions encode reasoning-related signals. They further decompose the complete adapter state with a cross-layer sparse autoencoder, uncovering monosemantic features that organize into categories like mathematical operators and reasoning markers, and perform an ablation study to quantify the contributions of MLP versus attention adapters. Overall, the results suggest that reasoning behavior can be elicited and examined through targeted, parameter-efficient interventions, offering a lens to understand LM dynamics and circuits with practical interpretability benefits.

Abstract

Reasoning models leverage inference-time compute to significantly enhance the performance of language models on difficult logical tasks, and have become a dominating paradigm in frontier LLMs. Despite their wide adoption, the mechanisms underpinning the enhanced performance of these reasoning models are not well understood. In this work, we show that the majority of new capabilities in reasoning models can be elicited by small, single-rank changes to base model parameters, with many of these changes being interpretable. Specifically, we use a rank-1 LoRA to create a minimal parameter adapter for Qwen-2.5-32B-Instruct which recovers 73-90% of reasoning-benchmark performance compared to a full parameter finetune. We find that the activations of this LoRA are as interpretable as MLP neurons, and fire for reasoning-specific behaviors. Finally, we train a sparse autoencoder on the entire activation state of this LoRA and identify fine-grained and monosemantic features. Our findings highlight that reasoning performance can arise largely from minimal changes to base model parameters, and explore what these changes affect. More broadly, our work shows that parameter-efficient training methods can be used as a targeted lens for uncovering fundamental insights about language model behavior and dynamics.

Figures (10)

• Figure 1: (left): Comparison of interpretability scores of individual LoRA adapter activations to arbitrarily sampled MLP neurons, and find that LoRA activations tend to be monosemantic roughly as often. (right): Comparison of autointerpretation categories between MLP neurons and LoRA adapter activations. LoRA adapters tend to activate more often for reasoning-specific feature categories.
• Figure 2: Overview of feature categories learned by an SAE trained on LoRA activation states. Percentages indicate relative feature activation densities. Categories and subcategories were generated by prompting an LLM with feature descriptions.
• Figure 3: Effect of ablating individual LoRA components from full adapter. (a) Effect of ablating all adapter components at a given layer on KL divergence. (b) Effect of ablating each adapter component individually. Ablation at later layers tends to have a significantly greater effect on the model's output distribution compared to earlier layers. MLP adapters, especially those trained on gate_proj matrices tend to have the strongest effect.
• Figure 4: Autointerpretation: Consistently activates on the token "Wait" (and surrounding commas/periods) across examples
• Figure 5: Autointerpretation: single-letter math variables and exponent tokens