Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ATLAS: Adaptive Test-Time Latent Steering with External Verifiers for Enhancing LLMs Reasoning

Tuc Nguyen, Thai Le

TL;DR

ATLAS introduces Adaptive Test-Time Latent Steering by coupling a lightweight external latent verifier with steering vectors to adapt reasoning decisions at inference. By extracting and contrasting middle-layer hidden-state representations across reasoning modes, ATLAS learns a per-step quality predictor and dynamically selects among execution, reflection, and transition interventions (or none) to guide the model efficiently. Empirical results across GSM8K, MATH500, and AIME show consistent accuracy gains and substantial token reductions, with larger models benefiting most from latent, scalable verification. The approach offers a practical, training-free augmentation to LLM reasoning that improves robustness and efficiency in both in-domain and cross-domain settings.

Abstract

Recent work on activation and latent steering has demonstrated that modifying internal representations can effectively guide large language models (LLMs) toward improved reasoning and efficiency without additional training. However, most existing approaches rely on fixed steering policies and static intervention strengths, which limit their robustness across problem instances and often result in over- or under-steering. We propose Adaptive Test-time Latent Steering, called (ATLAS), a task-specific framework that dynamically controls steering decisions at inference time using an external, lightweight latent verifier. Given intermediate hidden states, the verifier predicts the quality of ongoing reasoning and adaptively selects whether and how strongly to apply steering, enabling per-example and per-step adjustment with minimal overhead. To our knowledge, ATLAS is the first method to integrate learned latent verification into test-time steering for enhancing LLMs reasoning. Experiments on multiple mathematical reasoning benchmarks show that ATLAS consistently outperforms both vanilla decoding and fixed steering baselines, achieving higher accuracy while substantially reducing test-time token usage. These results demonstrate that verifier-guided latent adaptation provides an effective and scalable mechanism for controlling reasoning efficiency without sacrificing solution quality. All source code will be publicly available.

ATLAS: Adaptive Test-Time Latent Steering with External Verifiers for Enhancing LLMs Reasoning

TL;DR

ATLAS introduces Adaptive Test-Time Latent Steering by coupling a lightweight external latent verifier with steering vectors to adapt reasoning decisions at inference. By extracting and contrasting middle-layer hidden-state representations across reasoning modes, ATLAS learns a per-step quality predictor and dynamically selects among execution, reflection, and transition interventions (or none) to guide the model efficiently. Empirical results across GSM8K, MATH500, and AIME show consistent accuracy gains and substantial token reductions, with larger models benefiting most from latent, scalable verification. The approach offers a practical, training-free augmentation to LLM reasoning that improves robustness and efficiency in both in-domain and cross-domain settings.

Abstract

Recent work on activation and latent steering has demonstrated that modifying internal representations can effectively guide large language models (LLMs) toward improved reasoning and efficiency without additional training. However, most existing approaches rely on fixed steering policies and static intervention strengths, which limit their robustness across problem instances and often result in over- or under-steering. We propose Adaptive Test-time Latent Steering, called (ATLAS), a task-specific framework that dynamically controls steering decisions at inference time using an external, lightweight latent verifier. Given intermediate hidden states, the verifier predicts the quality of ongoing reasoning and adaptively selects whether and how strongly to apply steering, enabling per-example and per-step adjustment with minimal overhead. To our knowledge, ATLAS is the first method to integrate learned latent verification into test-time steering for enhancing LLMs reasoning. Experiments on multiple mathematical reasoning benchmarks show that ATLAS consistently outperforms both vanilla decoding and fixed steering baselines, achieving higher accuracy while substantially reducing test-time token usage. These results demonstrate that verifier-guided latent adaptation provides an effective and scalable mechanism for controlling reasoning efficiency without sacrificing solution quality. All source code will be publicly available.
Paper Structure (31 sections, 6 equations, 4 figures, 5 tables)

This paper contains 31 sections, 6 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Step-by-step Reasoning in LLMs.
  4. Activation Engineering.
  5. Process Reward Models and Verification.
  6. Motivation
  7. Method
  8. Problem Formulation
  9. Transformer Decoding Process.
  10. Large Reasoning Models (LRMs).
  11. Reasoning Steering Vectors Extraction
  12. External Verifier Training
  13. Data Construction.
  14. Training External Verifier $\mathcal{V}$.
  15. Adaptive Steering Policy
  16. ...and 16 more sections

Figures (4)

  • Figure 1: An example illustrating repetitive reasoning of Qwen-1.5B on the GSM8K dataset while executing single reasoning mode on SEAL chen2025seal.
  • Figure 2: ATLAS pipeline: (Top) Offline extraction of contrastive steering vectors from execution, reflection, and transition reasoning modes. (Bottom) Test-time adaptive steering through lightweight latent verification of hidden states.
  • Figure 3: Layer-wise ablation study of steering effectiveness. Dark blue indicates the accuracy achieved by ATLAS(L), while sky blue denotes the corresponding test-time token usage.
  • Figure 4: Pass@K performance of adaptive steering methods on the GSM8K dataset using R1-Distill-1.5B.