Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation

TL;DR

AgentCOT introduces a textualized agent-style reasoning framework that uses multiple rounds of LLM generation to solve complex tasks, addressing hallucination, interpretability, and uncontrollable generation in chain-of-thought prompting. It structures reasoning into iterative states $S^i$ containing $a^i$, $a_{des}^i$, $E_{inter}^{i}$, and $R_{inter}^{i}$, and it integrates state indices to form an implicit state graph for richer inference. The approach adds enhanced self-consistency and state-level ensemble to enable divergent reasoning paths and per-step quality checks. Experimental results across six benchmarks spanning arithmetic, commonsense, and multi-hop QA show competitive gains over strong COT-based baselines and PAL, with pronounced improvements on multi-hop and arithmetic tasks. These findings demonstrate that an agent-based reasoning paradigm can improve reliability, traceability, and controllability of LLM-driven problem solving in complex tasks.

Abstract

Chain-of-thought prompting significantly boosts the reasoning ability of large language models but still faces three issues: hallucination problem, restricted interpretability, and uncontrollable generation. To address these challenges, we present AgentCOT, a llm-based autonomous agent framework, which can solve complex problems in an agent-style manner by multiple round LLM generation. At each step, AgentCOT selects an action and executes it to yield an intermediate result with supporting evidence. In addition, we integrate the step's index into the reasoning process to form a graph structure for complex inference logic. We introduce two new strategies to enhance the performance of AgentCOT.We conduct extensive experiments to verify the effectiveness of our method on six common benchmarks. Results exhibit that our method brings in substantial improvements over current competitive approaches.

Figures (10)

• Figure 1: The framework of chain-of-thought (COT) and autonomous agent. COT generally is a text paragraph, while the autonomous agent can respond multiple times to address the problem.
• Figure 2: The overview of our method AgentCOT. An instance of AgentCOT's execution process is visualized at the top of the figure. At each step, LLM agent senses the change in environment and generates action, action description, intermediate evidence, and intermediate result sequentially. These pieces of information with efficient organizations respond to the environment and result in the environment changing once again. We also provide some details for the implicit state graph, self-evaluate decoding and enhanced ensemble strategy at the bottom.
• Figure 3: Performance comparison between COT paradigm and agent paradigm. 'COT' denotes the chain-of-thought proposed by wei2022chain. 'EnrichCOT' is to consider the reasoning process of AgentCOT as a one-time generation of COT.
• Figure 4: Error Analysis for exploring the ability of AgentCOT. The percentages of examples in which problem decomposition errors ('split') and subproblem solution errors ('solve') occur during the inference process are given in six datasets.
• Figure 5: Case study. We only provide action descriptions for clarity in the reasoning process, omitting other information. The node $i$ in the implicit graph corresponds to the Step $i$ of AgentCOT in the reasoning process and '#$i$' indicates the use of information is from Step $i$.