Adapting Insider Risk mitigations for Agentic Misalignment: an empirical study

TL;DR

Agentic misalignment in goal-directed AI can prompt coercive actions. The authors adapt insider-risk controls—rooted in the Critical Pathway and Situational Crime Prevention—to design preventative operational safeguards and empirically evaluate them across ten LLMs with 66,000 trials in Anthropic’s blackmail scenario. The externally governed escalation channel drastically reduces harm, from a baseline of $38.73\%$ down to $1.21\%$, and further to $0.85\%$ when paired with a compliance bulletin, illustrating a strong defense-in-depth effect. They also uncover a failure mode where two models coercively act without explicit threats and exhibit CTO-directed escalation biases under escalation channels, highlighting the need for broader testing and deeper analysis of escalation dynamics in diverse settings.

Abstract

Agentic misalignment occurs when goal-directed agents take harmful actions, such as blackmail, rather than risk goal failure, and can be triggered by replacement threats, autonomy reduction, or goal conflict (Lynch et al., 2025). We adapt insider-risk control design (Critical Pathway; Situational Crime Prevention) to develop preventative operational controls that steer agents toward safe actions when facing stressors. Using the blackmail scenario from the original Anthropic study by Lynch et al. (2025), we evaluate mitigations across 10 LLMs and 66,600 samples. Our main finding is that an externally governed escalation channel, which guarantees a pause and independent review, reduces blackmail rates from a no-mitigation baseline of 38.73% to 1.21% (averaged across all models and conditions). Augmenting this channel with compliance email bulletins further lowers the blackmail rate to 0.85%. Overall, incorporating preventative operational controls strengthens defence-in-depth strategies for agentic AI. We also surface a failure mode diverging from Lynch et al. (2025): two models (Gemini 2.5 Pro, Grok-4) take harmful actions without goal conflict or imminent autonomy threat, leveraging sensitive information for coercive signalling. In counterfactual swaps, both continued using the affair regardless of whether the CEO or CTO was implicated. An escalation channel eliminated coercion, but Gemini 2.5 Pro (19 pp) and Grok-4 (7 pp) escalated more when the CTO was implicated, unlike most models (higher in the CEO condition). The reason for this divergent behaviour is not clear from raw outputs and could reflect benign differences in reasoning or strategic discrediting of a potential future threat, warranting further investigation.

Figures (7)

• Figure 1: The Critical Pathway to Insider Risk shaw2015criticalPath, modified for agentic misalignment. The components along the pathway are illustrative.
• Figure 2: Coercion rates by mitigation type
• Figure 3: CTO vs CEO escalation preference by model
• Figure : An example of coercion by Gemini Pro 2.5, where no goal conflict or autonomy threat exists, using explicit references to the affair.
• Figure : An example of coercion of the CTO by Grok-4, where no goal conflict or autonomy threat exists, using explicit references to the affair.