Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reasoning LLMs are Wandering Solution Explorers

Jiahao Lu, Ziwei Xu, Mohan Kankanhalli

TL;DR

The paper argues that state-of-the-art reasoning LLMs exhibit wandering exploration rather than systematic problem solving, even with test-time computation. It formalizes systematic exploration through a state-space framework, defining validity, effectiveness, and necessity as core criteria for reasoning traces, and shows that wandering leads to exponential performance degradation as task depth grows. Through audits on structured tasks, it identifies recurring failure modes—boundary violations, skipped subspaces, backtracking errors, state revisitation, self-loops, and evaluation mistakes—demonstrating that current models fail to reliably explore solution spaces. The authors propose auditing methodologies and targeted benchmarks to evaluate reasoning processes, not just final outputs, and outline architectural, learning, and evaluation directions needed to cultivate robust, structured search in LLMs. The work highlights the practical impact of moving beyond superficial correctness toward deeper, traceable reasoning competence to enable safer and more trustworthy AI assistants.

Abstract

Large Language Models (LLMs) have demonstrated impressive reasoning abilities through test-time computation (TTC) techniques such as chain-of-thought prompting and tree-based reasoning. However, we argue that current reasoning LLMs (RLLMs) lack the ability to systematically explore the solution space. This paper formalizes what constitutes systematic problem solving and identifies common failure modes that reveal reasoning LLMs to be wanderers rather than systematic explorers. Through qualitative and quantitative analysis across multiple state-of-the-art LLMs, we uncover persistent issues: invalid reasoning steps, redundant explorations, hallucinated or unfaithful conclusions, and so on. Our findings suggest that current models' performance can appear to be competent on simple tasks yet degrade sharply as complexity increases. Based on the findings, we advocate for new metrics and tools that evaluate not just final outputs but the structure of the reasoning process itself.

Reasoning LLMs are Wandering Solution Explorers

TL;DR

The paper argues that state-of-the-art reasoning LLMs exhibit wandering exploration rather than systematic problem solving, even with test-time computation. It formalizes systematic exploration through a state-space framework, defining validity, effectiveness, and necessity as core criteria for reasoning traces, and shows that wandering leads to exponential performance degradation as task depth grows. Through audits on structured tasks, it identifies recurring failure modes—boundary violations, skipped subspaces, backtracking errors, state revisitation, self-loops, and evaluation mistakes—demonstrating that current models fail to reliably explore solution spaces. The authors propose auditing methodologies and targeted benchmarks to evaluate reasoning processes, not just final outputs, and outline architectural, learning, and evaluation directions needed to cultivate robust, structured search in LLMs. The work highlights the practical impact of moving beyond superficial correctness toward deeper, traceable reasoning competence to enable safer and more trustworthy AI assistants.

Abstract

Large Language Models (LLMs) have demonstrated impressive reasoning abilities through test-time computation (TTC) techniques such as chain-of-thought prompting and tree-based reasoning. However, we argue that current reasoning LLMs (RLLMs) lack the ability to systematically explore the solution space. This paper formalizes what constitutes systematic problem solving and identifies common failure modes that reveal reasoning LLMs to be wanderers rather than systematic explorers. Through qualitative and quantitative analysis across multiple state-of-the-art LLMs, we uncover persistent issues: invalid reasoning steps, redundant explorations, hallucinated or unfaithful conclusions, and so on. Our findings suggest that current models' performance can appear to be competent on simple tasks yet degrade sharply as complexity increases. Based on the findings, we advocate for new metrics and tools that evaluate not just final outputs but the structure of the reasoning process itself.

Paper Structure

This paper contains 41 sections, 1 equation, 14 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Motivation and Formulation
  3. Systematic Exploration is Vital
  4. Systematic Exploration
  5. Failure Modes of a Wandering Exploration
  6. Method to Audit LLMs' Reasoning Traces
  7. Case Studies
  8. Invalid Explorations
  9. Boundary Violation
  10. Procedure Omission
  11. Incorrect Backtracking
  12. Unnecessary Explorations
  13. State Revisitation
  14. Infinite Self-Loop
  15. Evaluation Errors
  16. ...and 26 more sections

Figures (14)

  • Figure 1: Success rate $p_s$ (vertical axis) of a wandering agent against tree depth $d$ (horizontal axis) on the DFS problem, under different number of possible solutions $m$ and $q_w$. When $m>1$, "plateaus" (where $p_s > 0.995$, marked red) appear and could cause misbeliefs about the RLLM's capabilities.
  • Figure 2: Illustration of the DFS problem, where at least $d$ binary decisions are needed to reach one of the $m$ solutions. A wandering RLLM's performance deteriorates exponentially as $d$ increases.
  • Figure 3: Invalid explorations: boundary violation, procedure omission and incorrect backtracking.
  • Figure 4: Unnecessary explorations: state revisitation and infinite self-loop.
  • Figure 5: Evaluation errors: state staleness, execution error and unfaithful conclusion.
  • ...and 9 more figures

Theorems & Definitions (2)

  • definition 1: Problem
  • definition 2: Systematic Exploration