From Task Solving to Robust Real-World Adaptation in LLM Agents

TL;DR

This work advances the evaluation of LLM-based agents by introducing a controllable grid-world benchmark that deliberately breaks clean-interface assumptions with partial observability, dynamic shifts, noisy sensing, and internal drift. The Grid Game requires long-horizon planning, information gathering, and cautious interaction to collect $K=3$ keys and reach $D$, while exposing hidden rules through latent tiles and costly probes. Across five modern models, it reveals a persistent gap between nominal task solving and deployment-like robustness, with performance sensitive to grid size and uncertainty regime and strategy often trumping raw capability. The findings motivate future work in uncertainty-aware verification, online change detection, and multi-objective training that explicitly balances completion, efficiency, and safety under non-stationarity.

Abstract

Large language models are increasingly deployed as specialized agents that plan, call tools, and take actions over extended horizons. Yet many existing evaluations assume a "clean interface" where dynamics are specified and stable, tools and sensors are reliable, and success is captured by a single explicit objective-often overestimating real-world readiness. In practice, agents face underspecified rules, unreliable signals, shifting environments, and implicit, multi-stakeholder goals. The challenge is therefore not just solving tasks, but adapting while solving: deciding what to trust, what is wanted, when to verify, and when to fall back or escalate. We stress-test deployment-relevant robustness under four operational circumstances: partial observability, dynamic environments, noisy signals, and dynamic agent state. We benchmark agentic LLMs in a grid-based game with a simple goal but long-horizon execution. Episodes violate clean-interface assumptions yet remain solvable, forcing agents to infer rules, pay for information, adapt to environmental and internal shifts, and act cautiously under noise. Across five state-of-the-art LLM agents, we find large gaps between nominal task-solving and deployment-like robustness. Performance generally degrades as grid size and horizon increase, but rankings are unstable: weaker models can beat stronger ones when strategy matches the uncertainty regime. Despite no explicit instruction, agents trade off completion, efficiency, and penalty avoidance, suggesting partial objective inference. Ablations and feature analyses reveal model-specific sensitivities and failure drivers, motivating work on verification, safe action selection, and objective inference under partial observability, noise, and non-stationarity.

Figures (10)

• Figure 1: Experimental vs. real-world deployment for LLM agents. Agents often succeed in controlled experimental settings (fixed rules, clean inputs, frozen environment) but degrade in the real world due to partial observability, dynamic environments, noisy inputs, and changing agent behavior.
• Figure 2: A 9$\times$9 gridworld with partial observability: the agent observes only the centered 5$\times$5 window.
• Figure 3: Action-frequency profiles over the first $T{=}200$ steps under the full modifier setting. We plot only non-movement actions (Interact, Measure, Scan). Early spikes reflect information gathering and experimentation, while later low mass indicates trajectories dominated by movement.
• Figure 4: Single-stressor ablations where all modifiers are disabled except the one being swept. Each panel sweeps one factor (Noise, Latent, Hazard-Spread, Teleport-Step) and reports win rate per model, averaged over 5 random episodes per data point, isolating sensitivity to individual uncertainty and non-stationarity sources.
• Figure 5: Logistic regression attributions for predicting Win vs. Lose per LLM. Each cell is a coefficient on one of nine normalized features (positive increases win likelihood), trained per model on all episodes pooled from main results and ablations.