Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization

Nathan Egbuna, Saatvik Gaur, Sunishchal Dev, Ashwinee Panda, Maheep Chaudhary

TL;DR

Amortized Latent Steering (ALS) converts expensive test-time optimization into a single offline computation by deriving a steering vector $\mathbf{v} = \mathbb{E}[h_{\text{correct}}] - \mathbb{E}[h_{\text{incorrect}}]$ from successful versus failed reasoning. During inference, ALS applies a lightweight intervention $h_t' = h_t + \alpha \mathbf{v}$ whenever $\cos(h_t, \mathbf{v})$ falls below $\tau$, yielding constant-time updates with negligible overhead. Across GSM8K and MATH-500, ALS achieves $2$-$5\times$ speedups over iterative latent methods while matching or surpassing greedy CoT and Self-Consistency baselines, with up to a $101\%$ improvement in efficiency--accuracy trade-off on challenging tasks. The results suggest much of latent-optimization benefits can be captured offline, enabling production deployment of sophisticated reasoning systems, albeit with architecture-dependent effects and reliance on offline ground-truth data. Code is available at the provided URL.

Abstract

Test-time optimization remains impractical at scale due to prohibitive inference costs--techniques like iterative refinement and multi-step verification can require $10-100\times$ more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model's hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-500 benchmarks, ALS achieves $2-5\times$ speedup over iterative methods while matching or surpassing greedy Chain-of-Thought (CoT) and Self-Consistency baselines, yielding up to 101% improvement in efficiency--accuracy trade-off. These results show that much of latent optimization's benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at https://github.com/negbuna/ALS.

Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization

TL;DR

Amortized Latent Steering (ALS) converts expensive test-time optimization into a single offline computation by deriving a steering vector from successful versus failed reasoning. During inference, ALS applies a lightweight intervention whenever falls below , yielding constant-time updates with negligible overhead. Across GSM8K and MATH-500, ALS achieves - speedups over iterative latent methods while matching or surpassing greedy CoT and Self-Consistency baselines, with up to a improvement in efficiency--accuracy trade-off on challenging tasks. The results suggest much of latent-optimization benefits can be captured offline, enabling production deployment of sophisticated reasoning systems, albeit with architecture-dependent effects and reliance on offline ground-truth data. Code is available at the provided URL.

Abstract

Test-time optimization remains impractical at scale due to prohibitive inference costs--techniques like iterative refinement and multi-step verification can require more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model's hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-500 benchmarks, ALS achieves speedup over iterative methods while matching or surpassing greedy Chain-of-Thought (CoT) and Self-Consistency baselines, yielding up to 101% improvement in efficiency--accuracy trade-off. These results show that much of latent optimization's benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at https://github.com/negbuna/ALS.

Paper Structure

This paper contains 17 sections, 3 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Steering Vector Construction
  5. Test-Time Intervention
  6. Structured Output Variant
  7. Experimention and Results
  8. Experimental Setup
  9. Models and Datasets
  10. Baselines and Evaluation
  11. Task Difficulty and Model Sensitivity
  12. Ablation Studies
  13. Mechanism Analysis
  14. Conclusion
  15. Limitations.
  16. ...and 2 more sections

Figures (3)

  • Figure 2: ALS workflow. Offline, the model computes a steering vector to nudge hidden states $\{h_t\}$ toward successful reasoning trajectories.
  • Figure 3: Pareto frontier for ALS on GSM8K. Each point corresponds to a steering strength $\alpha$, showing the trade-off between accuracy and inference time. The curve highlights how intermediate $\alpha$ values yield the best balance of efficiency and accuracy.
  • Figure 4: Pareto frontier for ALS on MATH-500. Results illustrate higher sensitivity to $\alpha$ compared to GSM8K, with both weak and strong steering outperforming moderate values depending on prompt style.