TIDE: Trajectory-based Diagnostic Evaluation of Test-Time Improvement in LLM Agents

TL;DR

The paper addresses the challenge of evaluating test-time improvement (TTI) in autonomous LLM agents, arguing that final success rates miss critical dynamics of learning during interaction. It introduces Test-time Improvement Diagnostic Evaluation (TIDE), a lightweight, agent- and environment-agnostic framework that decomposes TTI into three high-signal metrics: Area Under Variation ($AUV$), Loop Ratio ($LR$), and Memory Index ($MI$). Through formalization with a POMDP-based model, the framework quantifies optimization efficiency, behavior adaptation, and memory utility, and demonstrates that performance hinges on interaction dynamics and agent–environment matching rather than mere scale. The empirical analysis across diverse environments shows that extremely large models often provide robust baselines, while adaptive models exploit working memory with minimal looping, underscoring the need to optimize environment interaction strategies for genuine TTI gains. Overall, TIDE provides a principled lens to diagnose and improve how agents evolve during task interaction, guiding future design toward better dynamic optimization rather than static reasoning depth.

Abstract

Recent advances in autonomous LLM agents demonstrate their ability to improve performance through iterative interaction with the environment. We define this paradigm as Test-Time Improvement (TTI). However, the mechanisms under how and why TTI succeed or fail remain poorly understood, and existing evaluation metrics fail to capture their task optimization efficiency, behavior adaptation after erroneous actions, and the specific utility of working memory for task completion. To address these gaps, we propose Test-time Improvement Diagnostic Evaluation (TIDE), an agent-agnostic and environment-agnostic framework that decomposes TTI into three comprehensive and interconnected dimensions. The framework measures (1) the overall temporal dynamics of task completion and (2) identifies whether performance is primarily constrained by recursive looping behaviors or (3) by burdensome accumulated memory. Through extensive experiments across diverse agents and environments, TIDE highlights that improving agent performance requires more than scaling internal reasoning, calling for explicitly optimizing the interaction dynamics between the agent and the environment.

Figures (10)

• Figure 1: Overview of our trajectory-based diagnostic evaluation framework. (a) An agent completes tasks through multi-turn interaction with the environment. (b) Interaction trajectories are collected for diagnostic analysis. (c) TIDE provides a unified and interconnected diagnosis of TTI trajectories via three complementary metrics. AUV quantifies optimization efficiency by aggregating trapezoidal sub-areas along the trajectory; LR distinguishes loop-induced stagnation from behavioral adaptation; MI isolates and analyzes the contribution of working memory.
• Figure 2: SR curves on three environments. For comparison, we report both AUV and SR. We reveal that SR obscures underlying efficiency differences, and few interaction turns fail to exhibit the agent's TTI capability.
• Figure 3: The relationship between Loop Ratio and corresponding AUV for each task in FrozenLake.
• Figure 4: We report MI in FrozenLake and WebShop. More MI results can be found in Appendix \ref{['appendix:mri']}.
• Figure 5: Window size denotes the number of most recent interaction turns retained in memory.