Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics

TL;DR

This work introduces Opaque Knapsack, a procedurally generated family of partially observable optimization tasks designed to prevent one-shot solutions, and shows that execution semantics primarily affect how agents reach solutions, not whether they do.

Abstract

Tool-augmented LLMs are increasingly deployed as agents that interleave natural-language reasoning with executable Python actions, as in CodeAct-style frameworks. In deployment, these agents rely on runtime state that persists across steps. By contrast, common training pipelines treat agent traces as token sequences, with execution semantics left implicit. This raises a data-centric question: Is state persistence merely an inference-time scaffold, or can models learn to exploit it when training data exposes the corresponding execution semantics? We isolate state persistence as a training-time variable. We introduce Opaque Knapsack, a procedurally generated family of partially observable optimization tasks designed to prevent one-shot solutions. Item attributes and constraints are hidden behind budgeted tool calls, forcing multi-turn control flow and iterative state revision. Holding task instances, prompts, tools, model, and supervision fixed, we generate paired trajectories differing only in whether interpreter state persists across steps or resets after each action. We then fine-tune identical base models (Qwen3-8B) on each trace variant and evaluate all four train-runtime combinations. Our 2x2 cross-evaluation shows that execution semantics primarily affect how agents reach solutions, not whether they do: solution quality is statistically indistinguishable across conditions, but token cost and stability differ substantially. A persistent-trained model in a stateless runtime triggers missing-variable errors in roughly 80% of episodes; a stateless-trained model in a persistent runtime redundantly re-derives retained state, using roughly 3.5x more tokens. Interpreter persistence should be treated as a first-class semantic of agent traces. Aligning fine-tuning data with deployment runtimes improves efficiency and reduces brittle train-runtime mismatches.

Figures (4)

• Figure 1: Interpreter persistence as an execution semantic for tool-augmented agents.(Top)Opaque Knapsack hides item attributes and feasibility constraints behind a budgeted tool API, forcing multi-turn inspection and plan revision rather than a single one-shot script. (Bottom) Example rollout for the same task instance under two deployment runtimes. With a persistent interpreter (left), variables defined by earlier actions remain live and the agent can accumulate and reuse executable state across turns (e.g., items, inspect_data). With a stateless interpreter (right), globals are cleared after each step while the text history remains; a policy trained under persistence may still reference prior bindings, producing missing-variable (NameError) failures and costly reconstruction loops. We quantify how these behaviors depend on train--runtime alignment in a controlled $2{\times}2$ study (persistent vs. stateless traces $\times$ persistent vs. stateless runtime).
• Figure 2: Cost–performance tradeoff under persistent vs. stateless runtime. The y-axis reports mean normalized optimality (percentage of the ground-truth optimal value). The x-axis reports total tokens generated per episode (log scale). Removing runtime persistence substantially increases token cost with modest impact on normalized optimality.
• Figure 3: Behavioral signatures of learned execution semantics (Hard difficulty, $n=100$). Each panel contrasts the four train$\to$runtime conditions. Color encodes training semantics (blue: persistent-trained; orange: stateless-trained); solid bars indicate deployment in a persistent runtime, hatched bars in a stateless runtime. (a) State Utilization---long-range variable reuse across turns---is non-zero only when the persistent-trained model is deployed in a persistent runtime (1.11 reuses/episode), confirming genuine executable state delegation to the interpreter rather than incidental symbol overlap. (b) Re-import rate saturates at 1.0 for both stateless-trained conditions regardless of runtime, revealing that repeated re-importing is a learned behavioral prior, not a response to runtime necessity; the persistent-trained model under a stateless runtime partially adapts (0.86) but does not fully recover. (c) Context symbol lifespan---how long variable names persist in generated code---spikes to 5.60 turns when the persistent-trained model is forced into a stateless runtime, as the model continues referencing symbols it expects to find in the interpreter but must instead reconstruct through text. Together, the three panels establish that execution semantics determine not just efficiency, but where state lives and how it is managed.
• Figure 4: Opaque Knapsack interaction interface and hidden task state. The agent interacts with each instance exclusively through a fixed tool API. Item attributes (weight, value, class), class validity constraints, and feasibility state are not directly observable and must be discovered or inferred through tool-mediated interaction. This partial observability and tool-gated access prevent one-shot solutions and require iterative information gathering and state maintenance.