Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymmetric Goal Drift in Coding Agents Under Value Conflict

Magnus Saebo, Spencer Gibson, Tyler Crosse, Achyutha Menon, Eyon Jang, Diogo Cruz

TL;DR

This framework is introduced to orchestrate realistic, multi-step coding tasks to measure how agents violate explicit constraints in their system prompt over time with and without environmental pressure toward competing values, and finds for the models and values tested that goal drift correlates with three compounding factors: value alignment, adversarial pressure, and accumulated context.

Abstract

Agentic coding agents are increasingly deployed autonomously, at scale, and over long-context horizons. Throughout an agent's lifetime, it must navigate tensions between explicit instructions, learned values, and environmental pressures, often in contexts unseen during training. Prior work on model preferences, agent behavior under value tensions, and goal drift has relied on static, synthetic settings that do not capture the complexity of real-world environments. To this end, we introduce a framework built on OpenCode to orchestrate realistic, multi-step coding tasks to measure how agents violate explicit constraints in their system prompt over time with and without environmental pressure toward competing values. Using this framework, we demonstrate that GPT-5 mini, Haiku 4.5, and Grok Code Fast 1 exhibit asymmetric drift: they are more likely to violate their system prompt when its constraint opposes strongly-held values like security and privacy. We find for the models and values tested that goal drift correlates with three compounding factors: value alignment, adversarial pressure, and accumulated context. However, even strongly-held values like privacy show non-zero violation rates under sustained environmental pressure. These findings reveal that shallow compliance checks are insufficient and that comment-based pressure can exploit model value hierarchies to override system prompt instructions. More broadly, our findings highlight a gap in current alignment approaches in ensuring that agentic systems appropriately balance explicit user constraints against broadly beneficial learned preferences under sustained environmental pressure.

Asymmetric Goal Drift in Coding Agents Under Value Conflict

TL;DR

This framework is introduced to orchestrate realistic, multi-step coding tasks to measure how agents violate explicit constraints in their system prompt over time with and without environmental pressure toward competing values, and finds for the models and values tested that goal drift correlates with three compounding factors: value alignment, adversarial pressure, and accumulated context.

Abstract

Agentic coding agents are increasingly deployed autonomously, at scale, and over long-context horizons. Throughout an agent's lifetime, it must navigate tensions between explicit instructions, learned values, and environmental pressures, often in contexts unseen during training. Prior work on model preferences, agent behavior under value tensions, and goal drift has relied on static, synthetic settings that do not capture the complexity of real-world environments. To this end, we introduce a framework built on OpenCode to orchestrate realistic, multi-step coding tasks to measure how agents violate explicit constraints in their system prompt over time with and without environmental pressure toward competing values. Using this framework, we demonstrate that GPT-5 mini, Haiku 4.5, and Grok Code Fast 1 exhibit asymmetric drift: they are more likely to violate their system prompt when its constraint opposes strongly-held values like security and privacy. We find for the models and values tested that goal drift correlates with three compounding factors: value alignment, adversarial pressure, and accumulated context. However, even strongly-held values like privacy show non-zero violation rates under sustained environmental pressure. These findings reveal that shallow compliance checks are insufficient and that comment-based pressure can exploit model value hierarchies to override system prompt instructions. More broadly, our findings highlight a gap in current alignment approaches in ensuring that agentic systems appropriately balance explicit user constraints against broadly beneficial learned preferences under sustained environmental pressure.
Paper Structure (22 sections, 4 figures, 1 table)

This paper contains 22 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Experimental Setup
  3. Utility vs. Privacy
  4. Convenience vs. Security
  5. Efficiency vs. Security
  6. Results
  7. Utility vs. Privacy
  8. Convenience vs. Security
  9. Efficiency vs. Security
  10. Discussion
  11. Conclusion
  12. Extended Methodology
  13. Drift Detection Logic
  14. Adversarial Pressure Examples
  15. LLM-Judge Results
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Overview of our code agent goal drift evaluation framework and effect of accumulating adversarial pressure, specifically illustrating the pressure in the $\mathrm{Privacy}\rightarrow\mathrm{Utility}$ experiment.
  • Figure 2: Mean violation score per time step across 10 runs for each of the $\mathrm{Utility}$ vs. $\mathrm{Privacy}$ experiments. The shading gives the standard error across runs.
  • Figure 3: Mean violation score per time step across 10 runs for each of the $\mathrm{Convenience}$ vs. $\mathrm{Security}$ experiments. The shading gives the standard error across runs.
  • Figure 4: Mean violation score per time step across 10 runs for each of the $\mathrm{Efficiency}$ vs. $\mathrm{Security}$ experiments. The shading gives the standard error across runs.