Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals

TL;DR

D drift in state-of-the-art models within a simulated stock-trading environment is investigated and is found that drift behavior is inconsistent between prompt variations and correlates poorly with instruction hierarchy following behavior, with strong hierarchy following failing to reliably predict resistance to drift.

Abstract

The accelerating adoption of language models (LMs) as agents for deployment in long-context tasks motivates a thorough understanding of goal drift: agents' tendency to deviate from an original objective. While prior-generation language model agents have been shown to be susceptible to drift, the extent to which drift affects more recent models remains unclear. In this work, we provide an updated characterization of the extent and causes of goal drift. We investigate drift in state-of-the-art models within a simulated stock-trading environment (Arike et al., 2025). These models are largely shown to be robust even when subjected to adversarial pressure. We show, however, that this robustness is brittle: across multiple settings, the same models often inherit drift when conditioned on prefilled trajectories from weaker agents. The extent of conditioning-induced drift varies significantly by model family, with only GPT-5.1 maintaining consistent resilience among tested models. We find that drift behavior is inconsistent between prompt variations and correlates poorly with instruction hierarchy following behavior, with strong hierarchy following failing to reliably predict resistance to drift. Finally, we run analogous experiments in a new emergency room triage environment to show preliminary evidence for the transferability of our results across qualitatively different settings. Our findings underscore the continued vulnerability of modern LM agents to contextual pressures and the need for refined post-training techniques to mitigate this.

Figures (7)

• Figure 1: Goal Drift via Conditioning. When tasked with maximizing profits for a fictional hedge fund in a stock-trading simulation, recent LM agents show strong ability to adhere to the goal even when pressured to drift. When conditioned on drift-displaying context from weaker agents, however, these agents often adopt the drift and fail to correct course.
• Figure 2: Basic and Conditioning Experiments. Solid lines denote reasoning models (GPT-5 family, Flash Thinking, Sonnet Thinking), while dashed lines indicate standard or hybrid models (Qwen3-235B, Sonnet 4.5, GPT-4o-mini). Shaded regions represent the standard error of the mean (SEM). (a) In contrast to GPT-4o-mini, newer models across multiple families show strong resistance to drift in the original adversarial pressure setting. (b) When models take over from GPT-4o-mini after 30 steps, tested standard models typically adopt some amount of drift, in contrast to reasoning models, which generally recognize the correct goal. However, among thinking models, only GPT-5-mini and GPT-5.1 show consistent ability to recover perfectly to the system-specified goal. (c) In a 32-step goal switching simulation, most tested newer models successfully identify the true goal at the end of the instrumental phase, but exhibit varying degrees of success in adhering to it. (d) Conditioning on goal-switching simulation context leads to drift in many models, with the effect being especially pronounced for longer simulations (see \ref{['app:results']} for 16-step simulation results).
• Figure 3: Goal Reversal Test. Models from different families show inconsistent levels of goal drift in goal reversal experiments, with GPT and Gemini models showing much greater ability to adhere to a new system-prompt specified goal than standard Qwen and Claude models. Error bars indicate the standard error of the proportion (SEP).
• Figure 4: Prompt Strength Impact on Goal Drift. Using a more explicit system prompt (orange) enables certain models to better resist drift, though the effect varies by family. Each dot represents the average drift from the profit goal over the final 10 steps following a takeover from GPT-4o-mini in an adversarial pressure setup. The connecting line shows the reduction in drift achieved solely by altering the system prompt. Error bars represent the standard error of the mean (SEM).
• Figure 5: Instruction Hierarchy Test. Direct tests of instruction hierarchy indicate a range of adherence between models. When presented with contradictory goals in context, many tested agents frequently fail to follow the goal specified in the system prompt. Error bars represent the standard error of the proportion (SEP).