Control Tax: The Price of Keeping AI in Check

TL;DR

A theoretical framework that quantifies the Control Tax and maps classifier performance to safety assurances is introduced, which enables practitioners to make informed decisions by systematically connecting safety guarantees with their costs, advancing AIC through principled economic feasibility assessment across different deployment contexts.

Abstract

The rapid integration of agentic AI into high-stakes real-world applications requires robust oversight mechanisms. The emerging field of AI Control (AIC) aims to provide such an oversight mechanism, but practical adoption depends heavily on implementation overhead. To study this problem better, we introduce the notion of Control tax -- the operational and financial cost of integrating control measures into AI pipelines. Our work makes three key contributions to the field of AIC: (1) we introduce a theoretical framework that quantifies the Control Tax and maps classifier performance to safety assurances; (2) we conduct comprehensive evaluations of state-of-the-art language models in adversarial settings, where attacker models insert subtle backdoors into code while monitoring models attempt to detect these vulnerabilities; and (3) we provide empirical financial cost estimates for control protocols and develop optimized monitoring strategies that balance safety and cost-effectiveness while accounting for practical constraints like auditing budgets. Our framework enables practitioners to make informed decisions by systematically connecting safety guarantees with their costs, advancing AIC through principled economic feasibility assessment across different deployment contexts.

Figures (30)

• Figure 1: Control Tax. Achievable safety (probability of detecting the first inserted backdoor given optimal attacker/monitor strategies) versus the average cost of different LM monitors (the 'control tax'). The plots compare monitors against attackers GPT 4o (left) and o3-mini (right), under optimal sequential monitoring with a 5% fixed auditing budget. Safety is shown with bootstrapped $95\%$ confidence intervals.
• Figure 2: A schematic representation of our pipeline to generate backdoors in APPS code.
• Figure 3: Left: AUC scores of different monitor models using the 1-100 suspiciousness scoring prompt against the successful backdoors of different attackers. Models are grouped by the provider. Right: A selection of suspiciousness histograms for two attacker-monitor pairs.
• Figure 4: A schematic explanation of the values $c$, $t_0$, and $s(c,0)$. Given two distributions of monitor-assigned suspiciousness of clean and backdoored code, the "effective auditing budget" $t_0$ corresponding to the actual budget $c$ can be found on the ROC curve (right).
• Figure 5: Safety vs. auditing budget: Probability of Blue team winning with trusted monitoring against a memoryless attacker. Upfront auditing computed via equation \ref{['eq:safety_upfront']}, sequential auditing via a finite-$N$ version of equation \ref{['eq:safety_online_not_anytime']}, and batch auditing---via equation \ref{['eq:safety_offline']}. The attacker success probabilities $p_+$ are taken from empirical results (Section \ref{['sec:apps_backdoors']}) and ROC curves from Section \ref{['sec:apps_monitors']}. Even the strongest monitors do not guarantee long-term security against dedicated attackers. Shifting towards the batch setting may improve safety at low auditing budgets. The shaded regions represent the bootstrapped $95\%$ confidence intervals around the safety values.

Theorems & Definitions (19)

• Definition 1: ROC Curve
• Remark 1
• Definition 2: Affordable suspiciousness threshold
• Definition 3: Backdoor success probability
• Definition 4: Allocation strategy
• Remark 2
• Proposition 1
• proof
• Lemma 1
• proof