Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic

TL;DR

The paper argues that traditional test-time scaling, tied to generation length, fails in agentic LLM workflows where tool latency significantly affects total task time. It proposes Timely Machine, redefining test-time as wall-clock time, and introduces Timely-Eval to benchmark time-budget-aware reasoning across interactive games, ML tasks, and general reasoning. A cold-start SFT phase (Timely-Cold Start) is followed by Timely-RL, which uses a time-aware reward and a Timer Tool to train models that strategically allocate time within a given budget. Empirical results show that smaller models excel under fast tool feedback while larger models shine in high-latency environments, and Timely-RL consistently improves time-budget awareness and performance, offering a new lens for test-time scaling in the agentic era.

Abstract

As large language models (LLMs) increasingly tackle complex reasoning tasks, test-time scaling has become critical for enhancing capabilities. However, in agentic scenarios with frequent tool calls, the traditional generation-length-based definition breaks down: tool latency decouples inference time from generation length. We propose Timely Machine, redefining test-time as wall-clock time, where models dynamically adjust strategies based on time budgets. We introduce Timely-Eval, a benchmark spanning high-frequency tool calls, low-frequency tool calls, and time-constrained reasoning. By varying tool latency, we find smaller models excel with fast feedback through more interactions, while larger models dominate high-latency settings via superior interaction quality. Moreover, existing models fail to adapt reasoning to time budgets. We propose Timely-RL to address this gap. After cold-start supervised fine-tuning, we use reinforcement learning to enhance temporal planning. Timely-RL improves time budget awareness and consistently boosts performance across Timely-Eval. We hope our work offers a new perspective on test-time scaling for the agentic era.

Figures (4)

• Figure 1: Pipeline for Timely-RL
• Figure 2: Scaling experiments across four text-based interactive games (Zork1, Advent, Enchanter, Detective) using Qwen3 models (0.6B-32B parameters). All models are cold-started with the same dataset. Each row represents different tool call latency settings: No latency, Low ($\sim$2s), Medium ($\sim$10s), and High ($\sim$50s).
• Figure 3: Changes in reasoning length across different time budgets. The time budget is the model's original reasoning time scaled by factors such as $0.75\times$ and $1.0\times$.
• Figure 4: Scaling Experiments on ML tasks and Interactive Games.