RATT: A Thought Structure for Coherent and Correct LLM Reasoning

TL;DR

RATT introduces a thought-tree framework that unifies factual correctness with global planning by performing planning and lookahead at each reasoning step and integrating RAG-based fact-checking at nodes. It formalizes the optimization of a thought tree with iterative node generation, retrieval, and RAG-corrected integration to maximize task-specific objectives. Across code generation, creative writing, hallucination detection, and Game of 24, RATT yields substantial gains in factual grounding and logical coherence over CoT, ToT, and RAT baselines, demonstrating robust improvements in complex reasoning tasks. The approach significantly advances reliable LLM reasoning for knowledge-intensive, multi-step problems with potential impact on safety-critical applications and real-world decision-making.

Abstract

Large Language Models (LLMs) gain substantial reasoning and decision-making capabilities from thought structures. However, existing methods such as Tree of Thought and Retrieval Augmented Thoughts often fall short in complex tasks due to the limitations of insufficient local retrieval of factual knowledge and inadequate global selection of strategies. These limitations make it challenging for these methods to balance factual accuracy and comprehensive logical optimization effectively. To address these limitations, we introduce the Retrieval Augmented Thought Tree (RATT), a novel thought structure that considers both overall logical soundness and factual correctness at each step of the thinking process. Specifically, at every point of a thought branch, RATT performs planning and lookahead to explore and evaluate multiple potential reasoning steps, and integrate the fact-checking ability of Retrieval-Augmented Generation (RAG) with LLM's ability to assess overall strategy. Through this combination of factual knowledge and strategic feasibility, the RATT adjusts and integrates the thought tree structure to search for the most promising branches within the search space. This thought structure significantly enhances the model's coherence in logical inference and efficiency in decision-making, and thus increases the limit of the capacity of LLM to generate reliable inferences and decisions based on thought structures. A broad range of experiments on different types of tasks showcases that the RATT structure significantly outperforms existing methods in factual correctness and logical coherence.

Figures (7)

• Figure 1: Comparison of LLM thought structures. With optimized tree structure and RAG, our method is capable of reducing factual and logical errors.
• Figure 2: Framework of RATT. Given an input including a task, a prompt, and a set of documents, the LLM generates multiple thought nodes with different strategies and embeds the prompt and documents. Then, the model integrates the nodes into one optimized node. We use the embedding of the optimized node to retrieve and select relative documents. Finally, the LLM corrects possible mistakes and enriches the node with selected documents' information. This generation and optimization process iterates until reaching the maximum rounds of iteration.
• Figure 3: Comparison of code generation performance on HumanEval of different methods.
• Figure 4: Comparison of different methods in task Creative Writing.
• Figure 5: Improvement by different methods on GPT models in task Creative Writing.