ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimization

TL;DR

ThinkPilot tackles the inefficiencies and misalignment of large reasoning models by introducing a training-free framework that automatically discovers think-prefixes to steer model reasoning. It builds a taxonomy of six atomic reasoning behaviors and evolves prefixes through an evolution-inspired loop using selection, crossover, and mutation, guided by task-specific behavior preferences. Empirical results show broad gains in efficiency, safety, and instruction following, and ThinkPilot synergizes with training-based methods to yield further improvements. The work highlights the controllability of model thinking via behavior-guided prompts and proposes a foundation for aligning LRMs’ reasoning with diverse task demands, with deployment simplicity and potential extensions into dynamic intervention.

Abstract

Large Reasoning Models (LRMs) are powerful, but they still suffer from inefficient and off-target reasoning. Currently, training-free methods are limited to either rigid heuristics or descriptive, non-actionable analyses. In this paper, we introduce ThinkPilot, a training-free framework that automatically optimizes LRMs reasoning. It uses an evolutionary process to generate think-prefixes, which are instructions that evolve driven by a taxonomy of reasoning behaviors to guide models toward superior performance. Extensive experiments demonstrate ThinkPilot's broad effectiveness: it significantly improves the accuracy-length trade-off for efficient reasoning, drastically improves safety (for example, cutting the StrongREJECT score of DeepSeek-R1-Distill-Qwen-32B from 27.0% to 0.7), and enhances instruction following. It also synergizes with existing training-based methods. Our analysis reveals that think-prefixes can reliably control LRMs' reasoning behaviors, and that different tasks have strong preferences for specific behavioral distributions. By automatically identifying and eliciting these behaviors, ThinkPilot provides a generalizable framework for aligning LRMs reasoning with task demands. Data and code are available at https://github.com/teqkilla/ThinkPilot

Figures (6)

• Figure 1: The comparison between (a) vanilla thinking process and (b) ThinkPilot, which guides an LRM by iteratively optimizing think-prefixes based on reasoning behaviors. On the R1-Qwen-7B model, after two iterations, ThinkPilot improves accuracy by 3.4% while reducing average token usage by 20.6% on AIME 24.
• Figure 2: Overview of the ThinkPilot. The method optimizes think-prefixes through an evolutionary loop (A), where the evolution process (C) involves selection, crossover, and mutation, guided by the taxonomy of reasoning behaviors (B). The complete prompts for crossover and mutation can refer to Appendix \ref{['app:detailed prompt']}.
• Figure 3: Control Success Rate for reasoning behaviors on AMC 23 for R1-Qwen-7B.
• Figure 4: Reasoning behaviors distribution of top 10% think-prefixes in the QwQ-32B model on three tasks.
• Figure 5: Comparison of iterative optimization under different reasoning behavior guidance settings, evaluated on the QwQ-32B model on Instruction Following. The chart contrasts the Vanilla baseline and the full ThinkPilot method (All Behaviors) with three variations: ThinkPilot without guidance (w/o Behaviors), with non-preferred behaviors, and with preferred behaviors. The annotated arrows illustrate the performance changes under these different guidance settings.