SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search

TL;DR

SPIRAL reframes LLM-based planning as a grounded and reflective search by embedding three specialized agents—Planner, Simulator, and Critic—within Monte Carlo Tree Search. This tri-agent, memoryless yet self-correcting framework mitigates brittle linear reasoning and sparse rewards, achieving robust long-horizon planning. Empirical results on DailyLifeAPIs and HuggingFace show substantial gains over Chain-of-Thought baselines and several state-of-the-art agents, with notable improvements in accuracy and token efficiency. The work demonstrates that structured, semantics-aware feedback combined with world-grounded predictions yields practical improvements for autonomous tooling tasks, and provides reproducible artifacts and prompts for future research.

Abstract

Large Language Models (LLMs) often falter at complex planning tasks that require exploration and self-correction, as their linear reasoning process struggles to recover from early mistakes. While search algorithms like Monte Carlo Tree Search (MCTS) can explore alternatives, they are often ineffective when guided by sparse rewards and fail to leverage the rich semantic capabilities of LLMs. We introduce SPIRAL (Symbolic LLM Planning via Grounded and Reflective Search), a novel framework that embeds a cognitive architecture of three specialized LLM agents into an MCTS loop. SPIRAL's key contribution is its integrated planning pipeline where a Planner proposes creative next steps, a Simulator grounds the search by predicting realistic outcomes, and a Critic provides dense reward signals through reflection. This synergy transforms MCTS from a brute-force search into a guided, self-correcting reasoning process. On the DailyLifeAPIs and HuggingFace datasets, SPIRAL consistently outperforms the default Chain-of-Thought planning method and other state-of-the-art agents. More importantly, it substantially surpasses other state-of-the-art agents; for example, SPIRAL achieves 83.6% overall accuracy on DailyLifeAPIs, an improvement of over 16 percentage points against the next-best search framework, while also demonstrating superior token efficiency. Our work demonstrates that structuring LLM reasoning as a guided, reflective, and grounded search process yields more robust and efficient autonomous planners. The source code, full appendices, and all experimental data are available for reproducibility at the official project repository.

Figures (6)

• Figure 1: A comparison of a Vanilla Agent and the SPIRAL Agent on a conditional planning task. The Vanilla Agent follows a linear but flawed plan, while SPIRAL uses its grounded and reflective search to correctly handle the weather-based constraint before acting.
• Figure 2: An overview of the SPIRAL framework, where a tri-LLM symbolic architecture drives the MCTS loop. (1) Expansion: The Planner proposes a new action ($a_2$) to expand the search from the current state ($s_2$). (2) Simulation & Reflection: The Simulator provides a grounded observation ($o_3$) for the action, while the Critic generates a strategic reflection score ($\rho_{\text{ref}}$) evaluating the action's merit. (3) Backpropagation: A composite reward, calculated using the Critic's score, is then propagated up the tree to update node values and intelligently guide future selections.
• Figure 3: Comparison of average token usage per task across different models and methods. Our SPIRAL method consistently reduces token costs compared to CoT baselines on both datasets.
• Figure 4: Comparison of average API call usage per task across different models and methods. SPIRAL requires more API calls, reflecting its more complex reasoning process.
• Figure 5: Token Efficiency of SOTA methods. This plot shows the final cascaded accuracy achieved per 10,000 tokens consumed. A higher value indicates greater efficiency.