Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StateAct: Enhancing LLM Base Agents via Self-prompting and State-tracking

Nikolai Rozanov, Marek Rei

TL;DR

StateAct introduces a training-free base agent that combines self-prompting and chain-of-states to address long-context reasoning and goal adherence in LLM agents. By outputting and maintaining explicit goal, state, thought, and action at each step, StateAct achieves superior base-agent performance over ReAct across Alfworld, Textcraft, and Webshop, while also improving efficiency. The approach maintains compatibility with test-time scaling, improves state-tracking accuracy (estimated ~88% in Alfworld), and does not require external tools or additional training data. Open-sourced implementation highlights StateAct as a scalable, practical foundation for advancing LLM-based agents in diverse tasks.

Abstract

Large language models (LLMs) are increasingly used as autonomous agents, tackling tasks from robotics to web navigation. Their performance depends on the underlying base agent. Existing methods, however, struggle with long-context reasoning and goal adherence. We introduce StateAct, a novel and efficient base agent that enhances decision-making through (1) self-prompting, which reinforces task goals at every step, and (2) chain-of-states, an extension of chain-of-thought that tracks state information over time. StateAct outperforms ReAct, the previous best base agent, by over 10% on Alfworld, 30% on Textcraft, and 7% on Webshop across multiple frontier LLMs. We also demonstrate that StateAct can be used as a drop-in replacement for ReAct with advanced LLM agent methods such as test-time scaling, yielding an additional 12% gain on Textcraft. By improving efficiency and long-range reasoning without requiring additional training or retrieval, StateAct provides a scalable foundation for LLM agents. We open source our code to support further research at https://github.com/ai-nikolai/stateact .

StateAct: Enhancing LLM Base Agents via Self-prompting and State-tracking

TL;DR

StateAct introduces a training-free base agent that combines self-prompting and chain-of-states to address long-context reasoning and goal adherence in LLM agents. By outputting and maintaining explicit goal, state, thought, and action at each step, StateAct achieves superior base-agent performance over ReAct across Alfworld, Textcraft, and Webshop, while also improving efficiency. The approach maintains compatibility with test-time scaling, improves state-tracking accuracy (estimated ~88% in Alfworld), and does not require external tools or additional training data. Open-sourced implementation highlights StateAct as a scalable, practical foundation for advancing LLM-based agents in diverse tasks.

Abstract

Large language models (LLMs) are increasingly used as autonomous agents, tackling tasks from robotics to web navigation. Their performance depends on the underlying base agent. Existing methods, however, struggle with long-context reasoning and goal adherence. We introduce StateAct, a novel and efficient base agent that enhances decision-making through (1) self-prompting, which reinforces task goals at every step, and (2) chain-of-states, an extension of chain-of-thought that tracks state information over time. StateAct outperforms ReAct, the previous best base agent, by over 10% on Alfworld, 30% on Textcraft, and 7% on Webshop across multiple frontier LLMs. We also demonstrate that StateAct can be used as a drop-in replacement for ReAct with advanced LLM agent methods such as test-time scaling, yielding an additional 12% gain on Textcraft. By improving efficiency and long-range reasoning without requiring additional training or retrieval, StateAct provides a scalable foundation for LLM agents. We open source our code to support further research at https://github.com/ai-nikolai/stateact .
Paper Structure (62 sections, 2 equations, 6 figures, 10 tables)

This paper contains 62 sections, 2 equations, 6 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Base agents
  4. Extensions of base agents: RAG, fine-tuning, test-time-scaling
  5. Alternative methods: tools, hand-crafted rules, multi-agent
  6. State tracking in LLM-based agents
  7. Method
  8. Self prompting
  9. Chain of states as state-tracking
  10. An example trace
  11. Experimental setup
  12. Alfworld
  13. Webshop
  14. Textcraft
  15. In-context learning
  16. ...and 47 more sections

Figures (6)

  • Figure 1: Overview of LLM-agent methods and their components.
  • Figure 2: Example of StateAct outputs in Alfworld.
  • Figure 3: An example textual interaction in Alfworld (right) and corresponding 3D rendering (left).
  • Figure 4: An textual interaction in Webshop (right) and correspondingwebsite rendering (left).
  • Figure 5: State vs. No State, on the 134 test examples from Alfworld, using gpt-3.5-turbo-1106
  • ...and 1 more figures