Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents

Zehong Wang, Fang Wu, Hongru Wang, Xiangru Tang, Bolian Li, Zhenfei Yin, Yijun Ma, Yiyang Li, Weixiang Sun, Xiusi Chen, Yanfang Ye

TL;DR

This work identifies a fundamental gap between reasoning and planning in LLM-driven agents, showing that step-wise, locally scored decisions fail to sustain long-horizon coherence even in fully observable environments. It formalizes planning with a deterministic state-transition framework and proves that minimal reasoning-based strategies can be arbitrarily suboptimal for extended horizons, while even shallow lookahead improves outcomes. To address this, the authors propose FLARE (Future-aware Lookahead with Reward Estimation), combining explicit lookahead, trajectory-level value propagation, and receding-horizon commitment to revise early decisions based on downstream consequences. Across KGQA benchmarks and tool-use scenarios, FLARE yields robust improvements that often beat larger reasoning-based models, underscoring the practical importance of planning-centric agent design for long-horizon tasks.

Abstract

Large language model (LLM)-based agents exhibit strong step-by-step reasoning capabilities over short horizons, yet often fail to sustain coherent behavior over long planning horizons. We argue that this failure reflects a fundamental mismatch: step-wise reasoning induces a form of step-wise greedy policy that is adequate for short horizons but fails in long-horizon planning, where early actions must account for delayed consequences. From this planning-centric perspective, we study LLM-based agents in deterministic, fully structured environments with explicit state transitions and evaluation signals. Our analysis reveals a core failure mode of reasoning-based policies: locally optimal choices induced by step-wise scoring lead to early myopic commitments that are systematically amplified over time and difficult to recover from. We introduce FLARE (Future-aware Lookahead with Reward Estimation) as a minimal instantiation of future-aware planning to enforce explicit lookahead, value propagation, and limited commitment in a single model, allowing downstream outcomes to influence early decisions. Across multiple benchmarks, agent frameworks, and LLM backbones, FLARE consistently improves task performance and planning-level behavior, frequently allowing LLaMA-8B with FLARE to outperform GPT-4o with standard step-by-step reasoning. These results establish a clear distinction between reasoning and planning.

Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents

TL;DR

This work identifies a fundamental gap between reasoning and planning in LLM-driven agents, showing that step-wise, locally scored decisions fail to sustain long-horizon coherence even in fully observable environments. It formalizes planning with a deterministic state-transition framework and proves that minimal reasoning-based strategies can be arbitrarily suboptimal for extended horizons, while even shallow lookahead improves outcomes. To address this, the authors propose FLARE (Future-aware Lookahead with Reward Estimation), combining explicit lookahead, trajectory-level value propagation, and receding-horizon commitment to revise early decisions based on downstream consequences. Across KGQA benchmarks and tool-use scenarios, FLARE yields robust improvements that often beat larger reasoning-based models, underscoring the practical importance of planning-centric agent design for long-horizon tasks.

Abstract

Large language model (LLM)-based agents exhibit strong step-by-step reasoning capabilities over short horizons, yet often fail to sustain coherent behavior over long planning horizons. We argue that this failure reflects a fundamental mismatch: step-wise reasoning induces a form of step-wise greedy policy that is adequate for short horizons but fails in long-horizon planning, where early actions must account for delayed consequences. From this planning-centric perspective, we study LLM-based agents in deterministic, fully structured environments with explicit state transitions and evaluation signals. Our analysis reveals a core failure mode of reasoning-based policies: locally optimal choices induced by step-wise scoring lead to early myopic commitments that are systematically amplified over time and difficult to recover from. We introduce FLARE (Future-aware Lookahead with Reward Estimation) as a minimal instantiation of future-aware planning to enforce explicit lookahead, value propagation, and limited commitment in a single model, allowing downstream outcomes to influence early decisions. Across multiple benchmarks, agent frameworks, and LLM backbones, FLARE consistently improves task performance and planning-level behavior, frequently allowing LLaMA-8B with FLARE to outperform GPT-4o with standard step-by-step reasoning. These results establish a clear distinction between reasoning and planning.
Paper Structure (45 sections, 4 theorems, 43 equations, 7 figures, 7 tables, 1 algorithm)

This paper contains 45 sections, 4 theorems, 43 equations, 7 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Failure Modes of Long-Horizon Planning
  4. Empirical Analysis of Planning Failures
  5. Fundamental Limits of Reasoning-based Policy in Long-Horizon Planning
  6. Implications for Long-Horizon Decision Making
  7. Flare for Long-Horizon Planning
  8. Explicit Lookahead
  9. Value Propagation
  10. Discussion
  11. Experiments
  12. Experimental Setup
  13. Effectiveness
  14. Planning Dynamics
  15. Efficiency Analysis
  16. ...and 30 more sections

Key Result

Proposition 3.1

Let $\pi_{\text{greedy}}$ be a step-wise greedy policy defined by $\pi_{\text{greedy}}(s) \in \arg\max_{a \in \mathcal{A}(s)} \hat{u}(s,a)$. Suppose there exists a state $s \in \mathcal{S}$ and two actions $a,b \in \mathcal{A}(s)$ such that $\hat{u}(s,a) > \hat{u}(s,b)$. Then for any finite horizon Consequently, the suboptimality gap of $\pi_{\text{greedy}}$ can be made arbitrarily large.

Figures (7)

  • Figure 1: Planning performance and early myopia under increasing horizons.Top: Accuracy (Hits@1) on CWQ and WebQSP by required planning horizon. Performance degrades rapidly for single step (i.e., reasoning-based policy) and beam search as the horizon increases, while lookahead degrades more slowly. Bottom: Myopic trap selection at the first decision step. Step-wise scoring (single step and beam search) frequently selects actions with high immediate scores but poor long-term outcomes, whereas lookahead suppresses such traps via future-aware evaluation.
  • Figure 2: Error accumulation and recovery dynamics across decision paradigms.Top: Fraction of trajectories remaining on a correct prefix over decision steps. Step-wise scoring (single step and beam search) policies diverge early and irreversibly, while lookahead preserves correct prefixes substantially longer. Bottom: Success rate conditioned on the position of the first error. After early deviations, single step (i.e., reasoning-based policy) and beam search policies rarely recover, whereas Lookahead enables consistent recovery across all error positions.
  • Figure 3: Performance-budget trade-offs of different planning strategies on CWQ and WebQSP. Results are plotted against total token budget, reflecting the inherent computational costs of different planning strategies. Flare demonstrates superior scalability compared to beam search and lookahead baselines.
  • Figure 4: Planning behavior on ALFWorld (tool-use). We report the success rate and the first-error position for different planning strategies implemented on top of ReAct and Reflexion. Flare consistently achieves higher success rates and higher first-error positions, demonstrating more stable long-horizon planning behavior beyond knowledge graph traversal.
  • Figure 5: Overview of Flare via Monte Carlo Tree Search. (a) Planning proceeds in a receding-horizon manner. At each step, Flare expands candidate actions with pruning for efficiency, performs repeated simulations, evaluates trajectories using trajectory-level feedback, propagates returns backward to update action values, and executes the highest-valued action before replanning in the next state. (b) During simulation, actions are selected using a UCB-style rule to balance exploration and exploitation. (c) Trajectory-level evaluation is amortized with a bounded trajectory memory, which retrieves and reuses scores for similar trajectories to reduce evaluation cost. Overall, the design enforces explicit lookahead and value propagation, allowing future outcomes to shape early decisions.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Proposition 3.1
  • Proposition 3.2
  • Proposition 3.3
  • Proposition 2.1: Worst-Case Suboptimality of $k$-Lookahead
  • proof
  • proof
  • proof
  • proof