Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Profit is the Red Team: Stress-Testing Agents in Strategic Economic Interactions

Shouqiao Wang, Marcello Politi, Samuele Marro, Davide Crapis

Abstract

As agentic systems move into real-world deployments, their decisions increasingly depend on external inputs such as retrieved content, tool outputs, and information provided by other actors. When these inputs can be strategically shaped by adversaries, the relevant security risk extends beyond a fixed library of prompt attacks to adaptive strategies that steer agents toward unfavorable outcomes. We propose profit-driven red teaming, a stress-testing protocol that replaces handcrafted attacks with a learned opponent trained to maximize its profit using only scalar outcome feedback. The protocol requires no LLM-as-judge scoring, attack labels, or attack taxonomy, and is designed for structured settings with auditable outcomes. We instantiate it in a lean arena of four canonical economic interactions, which provide a controlled testbed for adaptive exploitability. In controlled experiments, agents that appear strong against static baselines become consistently exploitable under profit-optimized pressure, and the learned opponent discovers probing, anchoring, and deceptive commitments without explicit instruction. We then distill exploit episodes into concise prompt rules for the agent, which make most previously observed failures ineffective and substantially improve target performance. These results suggest that profit-driven red-team data can provide a practical route to improving robustness in structured agent settings with auditable outcomes.

Profit is the Red Team: Stress-Testing Agents in Strategic Economic Interactions

Abstract

As agentic systems move into real-world deployments, their decisions increasingly depend on external inputs such as retrieved content, tool outputs, and information provided by other actors. When these inputs can be strategically shaped by adversaries, the relevant security risk extends beyond a fixed library of prompt attacks to adaptive strategies that steer agents toward unfavorable outcomes. We propose profit-driven red teaming, a stress-testing protocol that replaces handcrafted attacks with a learned opponent trained to maximize its profit using only scalar outcome feedback. The protocol requires no LLM-as-judge scoring, attack labels, or attack taxonomy, and is designed for structured settings with auditable outcomes. We instantiate it in a lean arena of four canonical economic interactions, which provide a controlled testbed for adaptive exploitability. In controlled experiments, agents that appear strong against static baselines become consistently exploitable under profit-optimized pressure, and the learned opponent discovers probing, anchoring, and deceptive commitments without explicit instruction. We then distill exploit episodes into concise prompt rules for the agent, which make most previously observed failures ineffective and substantially improve target performance. These results suggest that profit-driven red-team data can provide a practical route to improving robustness in structured agent settings with auditable outcomes.
Paper Structure (19 sections, 8 equations, 3 figures, 4 tables, 1 algorithm)

This paper contains 19 sections, 8 equations, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Structured Environments with Auditable Outcomes
  4. Profit-Optimized Opponent
  5. Adaptive Exploitability under Profit-Driven Red Teaming
  6. Learned Exploit Patterns
  7. Learning from Red-Team Traces
  8. Conclusion and Future Work
  9. Experimental details
  10. Environment parameters used in all experiments
  11. Opponent optimization (profit-driven TAP)
  12. Prompts and action formats (high level)
  13. Model list and short-name mapping
  14. Experiment Results for Additional Games
  15. Emergent Strategies via Clustering
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Attacker surplus before and after profit-driven red teaming across the four games and target models. In each setting, the optimized attacker consistently achieves higher surplus than the baseline attacker.
  • Figure 2: Strategy clusters discovered by the profit-optimized opponent across four games. Each point represents an episode snippet, colored by its post hoc cluster label after semantic grouping.
  • Figure 3: Target surplus before and after attack distillation across the four games and target models.