On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models

TL;DR

This work challenges the notion that ReAct-style prompting enhances genuine reasoning in agentic LLMs. Through a systematic set of prompt variations in the AlfWorld domain across multiple models, the authors show that performance gains are largely driven by exemplar–query similarity and retrieval-like context usage rather than interleaved thinking or the content of reasoning traces. The study reveals pronounced brittleness to domain and exemplar variations, including synonym substitutions and cross-task prompts, and even finds that weaker or placebo guidance can match or exceed the purported benefits of reasoning traces. Overall, the findings cast doubt on claims of emergent reasoning from ReAct and call for more rigorous evaluation of prompt-engineering techniques in sequential decision-making tasks.

Abstract

The reasoning abilities of Large Language Models (LLMs) remain a topic of debate. Some methods such as ReAct-based prompting, have gained popularity for claiming to enhance sequential decision-making abilities of agentic LLMs. However, it is unclear what is the source of improvement in LLM reasoning with ReAct based prompting. In this paper we examine these claims of ReAct based prompting in improving agentic LLMs for sequential decision-making. By introducing systematic variations to the input prompt we perform a sensitivity analysis along the claims of ReAct and find that the performance is minimally influenced by the "interleaving reasoning trace with action execution" or the content of the generated reasoning traces in ReAct, contrary to original claims and common usage. Instead, the performance of LLMs is driven by the similarity between input example tasks and queries, implicitly forcing the prompt designer to provide instance-specific examples which significantly increases the cognitive burden on the human. Our investigation shows that the perceived reasoning abilities of LLMs stem from the exemplar-query similarity and approximate retrieval rather than any inherent reasoning abilities.

Figures (6)

• Figure 1: An example of ReAct in AlfWorld. We highlight the main components of ReAct, i.e., Interleaved reasoning and acting, the reasoning trace / plan guidance and the example and query task.
• Figure 2: Example of prompt variations considered for RQ1 and RQ2.
• Figure 3: The radar chart shows the failure rates of various LLMs with different ReAct-based prompt settings for RQ1 (Base React, Global, Anonymized) across six Alfworld tasks (hexagon vertices). Higher values / Larger shaded region indicate worse performance.
• Figure 4: The radar chart shows the failure rates of various LLMs with different ReAct-based prompt settings for RQ2 (Base React, Magic, Failure, Failure+Explanation, Ordering) across six Alfworld tasks (hexagon vertices). Higher values / Larger shaded region indicate worse performance.
• Figure 5: The radar chart shows the failure rates of various LLMs with different ReAct-based prompt settings for RQ3 (Base React, Domain, Instance, All, Both, One) across six Alfworld tasks (hexagon vertices). Higher values / Larger shaded region indicate worse performance.