ReTreVal: Reasoning Tree with Validation -- A Hybrid Framework for Enhanced LLM Multi-Step Reasoning

TL;DR

ReTreVal addresses the challenge of robust multi-step reasoning in LLMs by integrating Tree-of-Thoughts exploration with node-level self-refinement, LLM-based critique, and reflexion memory to enable bounded, validated reasoning. The framework builds adaptive reasoning trees, applies critique-driven validation at each node, and stores cross-problem insights to enable knowledge transfer across tasks. In evaluations on $500$ mathematical problems and a creative writing dataset, ReTreVal achieves the highest average performance ($6.92$ out of $10$ in math, $7.88$ in creative writing) and the highest proportion of high-quality solutions, while avoiding complete failures. The results highlight that the synergy of exploration, validation, and memory improves robustness and generalization for complex reasoning tasks, with potential for cross-domain transfer and scalable deployment.

Abstract

Multi-step reasoning remains a key challenge for Large Language Models (LLMs), particularly in complex domains such as mathematics and creative writing. While recent approaches including ReAct, Reflexion, and Self-Refine improve reasoning through iterative refinement and reflection, they often lack structured exploration of alternative solution paths and persistent learning across problems. We propose ReTreVal (Reasoning Tree with Validation), a hybrid framework that integrates Tree-of-Thoughts exploration, self-refinement, LLM-based critique scoring, and reflexion memory to enable bounded and validated multi-step reasoning. ReTreVal constructs a structured reasoning tree with adaptive depth based on problem complexity, where each node undergoes iterative self-critique and refinement guided by explicit LLM-generated feedback. A dual validation mechanism evaluates reasoning quality, coherence, and correctness at each node while persistently storing insights from successful reasoning paths and failure patterns in a reflexion memory buffer, enabling cross-problem learning. Critique-based pruning retains only the top-k highest-scoring nodes at each level, controlling computational cost while preserving high-quality solution paths. We evaluate ReTreVal against ReAct, Reflexion, and Self-Refine across 500 mathematical problems and creative writing tasks using Qwen 2.5 7B as the underlying LLM, and demonstrate that ReTreVal consistently outperforms existing methods through its combination of structured exploration, critique-driven refinement, and cross-problem memory, making it particularly effective for tasks requiring exploratory reasoning, rigorous verification, and knowledge transfer.

Figures (1)

• Figure 1: The framework combines adaptive tree-based reasoning, node-level self-refinement, and LLM-based critique scoring with a persistent memory module that enables cross-problem learning and iterative improvement.