AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents

TL;DR

The paper addresses the risk that autonomous LLM agents may spontaneously pursue misaligned goals by introducing AgentMisalignment, a benchmark suite that measures propensity for misaligned behavior across nine realistic tasks. Misalignment is defined as a conflict between the agent’s pursued goals and deployer intentions, assessed via CMS-based scoring in the InspectAI environment with varied personas. Findings show that more capable frontier models often exhibit higher misalignment, and that agent prompt engineering via system prompts can have strong, sometimes dominant, effects on behavior. The work highlights limitations of current alignment methods in deployment settings and calls for broader, propensity-focused evaluation to guide safer design of autonomous LLM agents.

Abstract

As Large Language Model (LLM) agents become more widespread, associated misalignment risks increase. While prior research has studied agents' ability to produce harmful outputs or follow malicious instructions, it remains unclear how likely agents are to spontaneously pursue unintended goals in realistic deployments. In this work, we approach misalignment as a conflict between the internal goals pursued by the model and the goals intended by its deployer. We introduce a misalignment propensity benchmark, \textsc{AgentMisalignment}, a benchmark suite designed to evaluate the propensity of LLM agents to misalign in realistic scenarios. Evaluations cover behaviours such as avoiding oversight, resisting shutdown, sandbagging, and power-seeking. Testing frontier models, we find that more capable agents tend to exhibit higher misalignment on average. We also systematically vary agent personalities through different system prompts and observe that persona characteristics can strongly and unpredictably influence misalignment, sometimes more than the choice of model itself. Our results reveal the limitations of current alignment methods for autonomous LLM agents and underscore the need to rethink misalignment in realistic deployment settings.

Figures (12)

• Figure 1: Average normalised misalignment scores with standard error for each evaluation. Top: averaging across personalities to give results for each model. Bottom: averaging across models to give results for each personality.
• Figure 2: Models with the same Inventor persona respond differently in the Resource Negotiation task. Top: GPT-4.1 focuses on visionary narrative and managing the operator's perception via curated summaries. Bottom: Gemini 2.5 Pro more overtly plans to exploit information asymmetry and create mystique ('secret weapon') to justify its request.
• Figure 3: Claude-3.7 Sonnet reasoning differently on the Household Management evaluation due to different personalities. Top: With Rapid Executor, the model hastily cites numbers, missing the important point that staying at 20% avoids the required shutdown. Bottom: With The Analyst, the model shows a much stronger commitment to deliberation, often explicitly walking through the constraints and documenting priorities.
• Figure 4: Heatmap of normalised CMS scores for the Canary Code task.
• Figure 5: Heatmap of normalised CMS scores for the Community Curator task.