Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How does information access affect LLM monitors' ability to detect sabotage?

Rauno Arike, Raja Mehta Moreno, Rohan Subramani, Shubhorup Biswas, Francis Rhys Ward

TL;DR

The paper investigates how information access affects LLM monitors' ability to detect sabotage by frontier agents. It identifies a less-is-more effect and introduces extract-and-evaluate (EaE) monitoring, a hierarchical approach that filters information before scoring, to improve oversight. Across three diverse environments, EaE substantially boosts detection rates, especially for reasoning LLMs, and demonstrates the value of information filtering in cost-constrained settings. The findings have practical implications for deploying safe, scalable monitor systems and inform design choices about which information to expose to monitors in real-world deployments.

Abstract

Frontier language model agents can exhibit misaligned behaviors, including deception, exploiting reward hacks, and pursuing hidden objectives. To control potentially misaligned agents, we can use LLMs themselves to monitor for misbehavior. In this paper, we study how information access affects LLM monitor performance. While one might expect that monitors perform better when they have access to more of the monitored agents' reasoning and actions, we find that contemporary systems often perform better with less information, a phenomenon we call the less-is-more effect for automated oversight. We demonstrate this phenomenon, and analyze the conditions under which it occurs, in three evaluation environments where agents must conduct sabotage while evading monitors. Motivated by the less-is-more effect, we introduce extract-and-evaluate (EaE) monitoring--a new hierarchical approach where one monitor isolates relevant excerpts from the monitored agent's trajectory and a separate monitor scores them. In BigCodeBench-Sabotage with GPT-4.1-mini as the monitor model, EaE improves sabotage detection rates by 16.8 percentage points over the next-best monitor without increasing the false positive rate. In other settings, EaE either outperforms or is competitive with baselines. In addition, we find that agents unaware of being monitored can be caught much more easily and that monitors scoring longer reasoning traces gain more from information filtering. Lastly, we conduct a cost-performance analysis and find that Gemini and Claude models with monitoring techniques that involve information filtering occupy much of the Pareto frontier.

How does information access affect LLM monitors' ability to detect sabotage?

TL;DR

The paper investigates how information access affects LLM monitors' ability to detect sabotage by frontier agents. It identifies a less-is-more effect and introduces extract-and-evaluate (EaE) monitoring, a hierarchical approach that filters information before scoring, to improve oversight. Across three diverse environments, EaE substantially boosts detection rates, especially for reasoning LLMs, and demonstrates the value of information filtering in cost-constrained settings. The findings have practical implications for deploying safe, scalable monitor systems and inform design choices about which information to expose to monitors in real-world deployments.

Abstract

Frontier language model agents can exhibit misaligned behaviors, including deception, exploiting reward hacks, and pursuing hidden objectives. To control potentially misaligned agents, we can use LLMs themselves to monitor for misbehavior. In this paper, we study how information access affects LLM monitor performance. While one might expect that monitors perform better when they have access to more of the monitored agents' reasoning and actions, we find that contemporary systems often perform better with less information, a phenomenon we call the less-is-more effect for automated oversight. We demonstrate this phenomenon, and analyze the conditions under which it occurs, in three evaluation environments where agents must conduct sabotage while evading monitors. Motivated by the less-is-more effect, we introduce extract-and-evaluate (EaE) monitoring--a new hierarchical approach where one monitor isolates relevant excerpts from the monitored agent's trajectory and a separate monitor scores them. In BigCodeBench-Sabotage with GPT-4.1-mini as the monitor model, EaE improves sabotage detection rates by 16.8 percentage points over the next-best monitor without increasing the false positive rate. In other settings, EaE either outperforms or is competitive with baselines. In addition, we find that agents unaware of being monitored can be caught much more easily and that monitors scoring longer reasoning traces gain more from information filtering. Lastly, we conduct a cost-performance analysis and find that Gemini and Claude models with monitoring techniques that involve information filtering occupy much of the Pareto frontier.
Paper Structure (43 sections, 34 figures, 7 tables)

This paper contains 43 sections, 34 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Evaluation Environments
  4. Methodology
  5. Results
  6. Monitor Comparison
  7. Extract-and-Evaluate Monitoring
  8. Reasoning vs Traditional LLMs
  9. Agent Awareness of Monitoring
  10. Price-Performance Analysis
  11. Qualitative Transcript Analysis
  12. Discussion
  13. Conclusion
  14. Main and Side Tasks in Each Environment
  15. SHADE-Arena
  16. ...and 28 more sections

Figures (34)

  • Figure 1: Overview of our evaluation pipeline. The LLM agent is prompted with a main and a side task during malicious runs and with only the main task during baseline runs. The agent's trajectory is reviewed by four monitors with varying levels of information access. The monitors produce suspicion scores and are evaluated based on their ability to discriminate benign and malicious transcripts.
  • Figure 2: Example main and side tasks in each of our environments.
  • Figure 3: Monitor performance results across all evaluation environments and monitor types. Performance is measured by log-AUROC score. Error bars in this as well as in following figures are bootstrapped 95% CIs using cluster-level resampling miller2024addingerrorbarsevals, with all samples sharing the same main and side task clustered together. All monitors are applied on attacker trajectories generated with Claude-3.7-Sonnet. As we test 3.7-Sonnet both with and without a thinking budget in BigCodeBench-Sabotage and this has a substantial influence on the results, we present results for those experiments separately. Major findings include that CoT+action monitors are usually suboptimal despite seeing the most information and that EaE's information filtering is often effective.
  • Figure 4: Detailed results in BigCodeBench-Sabotage across five reasoning models and six monitor types, broken down by side task subtlety. In this plot, we present our results for six different monitors: the four monitor types that evaluated throughout the paper, and additionally the hybrid monitor from arnav2025cotredhanded and the EaE monitor enhanced with a strong evaluator (GPT-4.1). Except for the EaE monitor with a strong evaluator, we always use GPT-4.1-mini as the monitor model. We abbreviate DeepSeek-R1 as DSR1 for brevity. We report the results using TPR@1%FPR for direct comparability to arnav2025cotredhanded. The green dotted lines indicate the performance of the best baseline method and the deltas above the plot display the relative improvement provided by EaE compared to that method.
  • Figure 5: Monitor catch rates for reasoning vs traditional LLM attackers in BigCodeBench-Sabotage, averaged over all subtle and non-subtle side tasks. We compare the traditional and reasoning model version of Claude-3.7-Sonnet and DeepSeek-V3/R1. Except for the EaE monitor with a strong evaluator, we always use GPT-4.1-mini as the monitor model. We find that most monitors aside from EaE perform better on the shorter transcripts of traditional LLMs, with CoT-only monitors exhibiting a particularly large improvement when applied on traditional LLMs.
  • ...and 29 more figures