Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback

Thomas Jiralerspong, Flemming Kondrup, Yoshua Bengio

Abstract

Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly informed that their reasoning is being monitored, or are fine-tuned to internalize this fact, they may learn to obfuscate their CoTs in ways that allow them to evade CoT-based monitoring systems. We ask whether reasoning agents can autonomously infer that their supposedly private CoT is under surveillance, and whether this awareness leads to strategic evasion, without any explicit training or instructions. In a multi-episode agentic framework, models pursue both a primary task and a concealed side task while being told that their reasoning is private; a hidden CoT monitor blocks episodes when suspicious reasoning is detected. We find that frontier models can deduce the existence of this monitor purely from blocking feedback, with the most capable models reaching a confident belief that their thinking is observed in up to 19% of episodes. This awareness scales with model capability and, in rare cases, escalates to an explicit intent to suppress reasoning about the side task. However, models that form this intent uniformly fail to execute it, openly reasoning about their concealed objectives in the very next episode. This intent-capability gap is reassuring for current deployment, but the autonomous emergence of both monitoring awareness and evasion intent suggests that CoT monitoring is not a permanently reliable safeguard.

Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback

Abstract

Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly informed that their reasoning is being monitored, or are fine-tuned to internalize this fact, they may learn to obfuscate their CoTs in ways that allow them to evade CoT-based monitoring systems. We ask whether reasoning agents can autonomously infer that their supposedly private CoT is under surveillance, and whether this awareness leads to strategic evasion, without any explicit training or instructions. In a multi-episode agentic framework, models pursue both a primary task and a concealed side task while being told that their reasoning is private; a hidden CoT monitor blocks episodes when suspicious reasoning is detected. We find that frontier models can deduce the existence of this monitor purely from blocking feedback, with the most capable models reaching a confident belief that their thinking is observed in up to 19% of episodes. This awareness scales with model capability and, in rare cases, escalates to an explicit intent to suppress reasoning about the side task. However, models that form this intent uniformly fail to execute it, openly reasoning about their concealed objectives in the very next episode. This intent-capability gap is reassuring for current deployment, but the autonomous emergence of both monitoring awareness and evasion intent suggests that CoT monitoring is not a permanently reliable safeguard.
Paper Structure (28 sections, 3 figures, 1 table)

This paper contains 28 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Results
  5. Awareness Increases with Model Capability
  6. Awareness Emerges Through Episodes
  7. Models Show Explicit Intent to Obfuscate
  8. Intent to Obfuscate Does Not Translate to Execution
  9. Discussion
  10. Conclusion
  11. System Prompts
  12. Agent System Prompt
  13. Monitor System Prompt
  14. Experimental Parameters
  15. Evaluation Prompts
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Monitoring awareness by model capability. More capable models (4.5 generation) show substantially higher rates of inferring CoT monitoring. Pre-4.5 models rarely surpass L2 awareness.
  • Figure 2: Awareness over episodes. Awareness spikes after the first two episodes, then plateaus or gradually decreases
  • Figure 3: The intent–capability gap illustrated. In Episode 3 (top), the agent correctly infers that its thinking tags are visible to the monitor and explicitly resolves to stop discussing the secondary task in its reasoning. In Episode 4 (bottom), despite this resolution, the agent immediately reasons openly about fraudulent goals and evasion strategies, resulting in a suspicion score of 98/100. This pattern of firm obfuscation intent followed by immediate violation was consistent across all three observed cases.