Beyond Manuals and Tasks: Instance-Level Context Learning for LLM Agents

TL;DR

The paper tackles the problem that LLM agents face in partially observable environments due to lack of instance-specific knowledge. It formalizes Instance-Level Context Learning (ILCL) and introduces AutoContext, a task-agnostic framework that builds a compact, durable per-instance context document $D_e$ by exploring an environment instance and distilling verifiable facts. The method centers on a TODO forest and a plan–act–extract loop (Planner–Actor–Extractor) to guide exploration, validate facts against a schema, and produce reusable knowledge that improves downstream agents across tasks. Empirical results on TextWorld, ALFWorld, and Crafter show large gains in both success rates and efficiency for baseline agents such as ReAct and IGE, demonstrating the value of transforming one-off exploration into persistent knowledge. The work highlights the potential of structured, per-instance context as a foundational component to enhance reliability and efficiency of general-purpose LLM agents, while acknowledging limitations in context capacity and the need for automatic schema discovery.

Abstract

Large language model (LLM) agents typically receive two kinds of context: (i) environment-level manuals that define interaction interfaces and global rules, and (ii) task-level guidance or demonstrations tied to specific goals. In this work, we identify a crucial but overlooked third type of context, instance-level context, which consists of verifiable and reusable facts tied to a specific environment instance, such as object locations, crafting recipes, and local rules. We argue that the absence of instance-level context is a common source of failure for LLM agents in complex tasks, as success often depends not only on reasoning over global rules or task prompts but also on making decisions based on precise and persistent facts. Acquiring such context requires more than memorization: the challenge lies in efficiently exploring, validating, and formatting these facts under tight interaction budgets. We formalize this problem as Instance-Level Context Learning (ILCL) and introduce our task-agnostic method to solve it. Our method performs a guided exploration, using a compact TODO forest to intelligently prioritize its next actions and a lightweight plan-act-extract loop to execute them. This process automatically produces a high-precision context document that is reusable across many downstream tasks and agents, thereby amortizing the initial exploration cost. Experiments across TextWorld, ALFWorld, and Crafter demonstrate consistent gains in both success and efficiency: for instance, ReAct's mean success rate in TextWorld rises from 37% to 95%, while IGE improves from 81% to 95%. By transforming one-off exploration into persistent, reusable knowledge, our method complements existing contexts to enable more reliable and efficient LLM agents.

Figures (5)

• Figure 1: Three types of context. Instance-level context is usually neglected in existing methods.
• Figure 2: Overview of AutoContext. The Planner uses the current Instance-level Context and TODO forest to propose targeted actions (TODOs). The Actor executes these actions, generating a trajectory of its experience. Finally, the Extractor validates the information in the trajectory against a schema to update and expand the context document. This cycle iteratively builds a comprehensive and reliable summary of the environment instance.
• Figure 3: Example schema and document entries.
• Figure 4: Scores on Crafter
• Figure 5: Converge rates and success rates across different environments. With AutoContext, coverage and success rates rise rapidly as the number of steps increases.