LUMINA: Long-horizon Understanding for Multi-turn Interactive Agents

TL;DR

The paper tackles the challenge of long-horizon multi-turn agentic tasks where large language models struggle, proposing LUMINA, an oracle counterfactual framework to quantify the importance of planning, state tracking, and history management. It operationalizes these ideas in procedurally generated environments (ListWorld, TreeWorld, GridWorld) that support precise oracle interventions and trajectory-level analysis, enabling controlled isolation of each skill’s contribution. Results show that planning consistently improves success rates across settings, but the benefits of other skills are environment- and model-size dependent, with larger models sometimes deriving smaller gains or even experiencing negative effects from certain interventions due to context dynamics. The work provides a structured approach to diagnosing bottlenecks in long-horizon AI agents and offers guidance for future improvements in both language models and multi-turn task environments, emphasizing the role of intervention-aware design in robust agent performance.

Abstract

Large language models can perform well on many isolated tasks, yet they continue to struggle on multi-turn, long-horizon agentic problems that require skills such as planning, state tracking, and long context processing. In this work, we aim to better understand the relative importance of advancing these underlying capabilities for success on such tasks. We develop an oracle counterfactual framework for multi-turn problems that asks: how would an agent perform if it could leverage an oracle to perfectly perform a specific task? The change in the agent's performance due to this oracle assistance allows us to measure the criticality of such oracle skill in the future advancement of AI agents. We introduce a suite of procedurally generated, game-like tasks with tunable complexity. These controlled environments allow us to provide precise oracle interventions, such as perfect planning or flawless state tracking, and make it possible to isolate the contribution of each oracle without confounding effects present in real-world benchmarks. Our results show that while some interventions (e.g., planning) consistently improve performance across settings, the usefulness of other skills is dependent on the properties of the environment and language model. Our work sheds light on the challenges of multi-turn agentic environments to guide the future efforts in the development of AI agents and language models.

Figures (6)

• Figure 1: Formulation.(a) Oracle-augmented history. Within multi-turn tasks, we study LLM-based agents $\pi_\theta$ when additionally assisted by an oracle. We can leverage one or more oracles to modify the history $h_t$ (context for language model). (b) GridWorld example. In this example, the agent needs to navigate from an initial 2D location to a goal location. We can use oracle ${\mathcal{O}}^{\text{state}}$ to summarize the current location (instead of the model reflectively reasoning at each turn). Similarly, we can also use ${\mathcal{O}}^{\text{plan}}$ to hint way points to reach the goal. (c) History pruning. Since we consider Markov decision processes, ${\mathcal{O}}^{\text{history}}$ can be used to rewrite the task description such that the actions can be taken independent to previous steps.
• Figure 2: Environments. In this work, we study the influence of oracle interventions in three unique environments. In all cases, the agent reasons (shown in gray italic) and performs an action (shown in monospace), and the environment provides minimal but sufficient feedback to help the agent progress towards the goal. (a) ListWorld: which requires modifying a python list using only pop(idx) actions; (b) TreeWorld: where the task is to iteratively search over a tree to find a specific node; and (c) GridWorld: where the agent needs to move from an initial location to a goal location.
• Figure 3: Success rate and step accuracy of Qwen3-8B and Qwen3-32B models by task horizon in ListWorld (left), TreeWorld (middle), and GridWorld (right).
• Figure 4: Influence of oracle interventions on ListWorld top, TreeWorld (middle), and GridWorld (bottom). Results are averaged over all horizon lengths. The labels indicate the active oracles (S: state tracking, P: planning, and H: history pruning).
• Figure 5: Influence of oracle interventions for Gemma 3 and GPT-4o on ListWorld. Results are averaged over all horizon lengths. The labels indicate the active oracles (S: state tracking, P: planning, and H: history pruning).