Controllable LLM Reasoning via Sparse Autoencoder-Based Steering

TL;DR

This paper tackles the challenge of controllable reasoning in large language models by using sparse autoencoders to disentangle strategy-related hidden states into interpretable features. It introduces SAE-Steering, a two-stage pipeline that first recalls features likely to influence strategy-specific tokens via logit contributions, then ranks them by actual control effectiveness through intervention experiments. Empirical results show SAE-Steering significantly outperforms baselines in steering accuracy (averaging over 15% improvement) and can redirect erroneous reasoning paths, yielding about a 7% absolute accuracy gain. The approach demonstrates robust cross-domain generalization (math to GPQA) and provides a promising direction for reliable, fine-grained control of LLM reasoning in practical applications.

Abstract

Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies (e.g. backtracking, cross-verification) during reasoning process, which improves their performance on complex tasks. Currently, reasoning strategies are autonomously selected by LRMs themselves. However, such autonomous selection often produces inefficient or even erroneous reasoning paths. To make reasoning more reliable and flexible, it is important to develop methods for controlling reasoning strategies. Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs' hidden states. To address this, we leverage Sparse Autoencoders (SAEs) to decompose strategy-entangled hidden states into a disentangled feature space. To identify the few strategy-specific features from the vast pool of SAE features, we propose SAE-Steering, an efficient two-stage feature identification pipeline. SAE-Steering first recalls features that amplify the logits of strategy-specific keywords, filtering out over 99\% of features, and then ranks the remaining features by their control effectiveness. Using the identified strategy-specific features as control vectors, SAE-Steering outperforms existing methods by over 15\% in control effectiveness. Furthermore, controlling reasoning strategies can redirect LRMs from erroneous paths to correct ones, achieving a 7\% absolute accuracy improvement.

Figures (7)

• Figure 1: An illustration of reasoning strategy control. By deliberately controlling the LRM's strategy selection, we can flexibly intervene and correct its reasoning path when a flaw emerges.
• Figure 2: (a) Overview of the SAE architecture. (b) Feature identification pipeline of SAE-Steering. Numbers below the arrows indicate the approximate count of features retained.
• Figure 3: Case study: SAE-Steering changes reasoning behavior while Logit Boosting only boosts keywords.
• Figure 4: Recalled features across layers.
• Figure 5: Control effectiveness across layers.