Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PILOT: Planning via Internalized Latent Optimization Trajectories for Large Language Models

Haoyu Zheng, Yun Zhu, Yuqian Yuan, Bo Yuan, Wenqiao Zhang, Siliang Tang, Jun Xiao

TL;DR

PILOT tackles the fragility of long-horizon reasoning in compact LLMs by internalizing strategic planning as latent guidance. It learns an instance-specific anchor via a lightweight Hyper-Network and injects it at a pivot layer to steer reasoning without modifying backbone weights, achieving near-zero decoding overhead. Across mathematical and coding benchmarks, PILOT delivers consistent gains, especially on complex tasks such as MATH500, and demonstrates robustness under distribution shifts and cross-domain transfer. The approach couples a construct-and-verify data pipeline with energy-aware injection to maintain manifold-consistent updates, offering a scalable path to reliable, fast internal planning in LLMs.

Abstract

Strategic planning is critical for multi-step reasoning, yet compact Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks. Our analysis reveals that LLMs possess latent reasoning capabilities that can be unlocked when conditioned on explicit plans from a teacher model; however, runtime reliance on external guidance is often impractical due to latency and availability constraints. To bridge this gap, we propose PILOT (Planning via Internalized Latent Optimization Trajectories), a non-invasive framework designed to internalize the strategic oversight of large models into intrinsic Latent Guidance. Instead of altering backbone weights, PILOT employs a lightweight Hyper-Network to synthesize a query-conditioned Latent Guidance vector. This vector acts as an internal steering mechanism, guiding the model's representations toward optimal reasoning paths. Extensive experiments on mathematical and coding benchmarks demonstrate that PILOT effectively stabilizes reasoning trajectories, consistently outperforming strong baselines (e.g., +8.9% on MATH500) with negligible inference latency.

PILOT: Planning via Internalized Latent Optimization Trajectories for Large Language Models

TL;DR

PILOT tackles the fragility of long-horizon reasoning in compact LLMs by internalizing strategic planning as latent guidance. It learns an instance-specific anchor via a lightweight Hyper-Network and injects it at a pivot layer to steer reasoning without modifying backbone weights, achieving near-zero decoding overhead. Across mathematical and coding benchmarks, PILOT delivers consistent gains, especially on complex tasks such as MATH500, and demonstrates robustness under distribution shifts and cross-domain transfer. The approach couples a construct-and-verify data pipeline with energy-aware injection to maintain manifold-consistent updates, offering a scalable path to reliable, fast internal planning in LLMs.

Abstract

Strategic planning is critical for multi-step reasoning, yet compact Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks. Our analysis reveals that LLMs possess latent reasoning capabilities that can be unlocked when conditioned on explicit plans from a teacher model; however, runtime reliance on external guidance is often impractical due to latency and availability constraints. To bridge this gap, we propose PILOT (Planning via Internalized Latent Optimization Trajectories), a non-invasive framework designed to internalize the strategic oversight of large models into intrinsic Latent Guidance. Instead of altering backbone weights, PILOT employs a lightweight Hyper-Network to synthesize a query-conditioned Latent Guidance vector. This vector acts as an internal steering mechanism, guiding the model's representations toward optimal reasoning paths. Extensive experiments on mathematical and coding benchmarks demonstrate that PILOT effectively stabilizes reasoning trajectories, consistently outperforming strong baselines (e.g., +8.9% on MATH500) with negligible inference latency.
Paper Structure (63 sections, 30 equations, 5 figures, 8 tables, 1 algorithm)

This paper contains 63 sections, 30 equations, 5 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Evolution of Chain-of-Thought and Verification Strategies
  4. Parameter-Efficient Adaptation and Latent Reasoning
  5. Representation Engineering and Activation Steering
  6. Preliminaries
  7. Notations
  8. Problem Formulation
  9. The PILOT Framework
  10. Target State Extraction
  11. Verification and Blind-Test.
  12. Homogeneous Target Projection.
  13. Anchor Adapter Architecture
  14. Dual-Channel Context Aggregation.
  15. Proto-Anchor Modulation.
  16. ...and 48 more sections

Figures (5)

  • Figure 1: The PILOT Framework Architecture.(Top) Stage I: Heuristic State Extraction extracting the optimized latent state $\mathbf{z}^*$ from verified expert trajectories. (Bottom) Stage II: Latent Anchor Synthesis during inference predicting $\hat{\mathbf{z}}$ from query tokens. (Right) The Anchor Adapter modulates a Proto-Anchor$\mathbf{P}$ via a Hyper-Network $\mathcal{H}_\theta$ and injects it into the backbone via energy-aligned injection.
  • Figure 2: Data Construction via Construct-and-Verify. We filter for hard instances where the base model fails zero-shot but succeeds with expert guidance $g_{\text{exp}}$. These verified triplets $(x, g_{\text{exp}}, y^*)$ form the training set $\mathcal{D}_{\text{train}}$.
  • Figure 3: Energy Alignment Dynamics (7B). Tracking injection vector $L_2$ norm. Left (Math): Raw energy naturally aligns with context. Right (Code): PILOT's alignment constrains wild fluctuations, preventing "embedding shock" and ensuring stability.
  • Figure 4: Injection Depth Sensitivity (Qwen-1.5B). Optimal pivots shift by task: Math peaks at the deepest layer (26), while Code peaks earlier (20).
  • Figure 5: Cosine similarity between base and anchored states. (a) Math: Layer 26 anchoring indicates "last-mile" correction. (b) Code: Layer 20 injection triggers "shock" followed by recovery, implying deep restructuring. Shaded regions: std dev ($N=100$).