Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AI

TL;DR

The paper tackles safety risks from untrusted AI in high-stakes settings by proposing factored cognition protocols that decompose tasks into simpler subproblems solved under trusted monitoring, with reassembly into a full solution. It evaluates two variants, Factor($T$,$U$) and Factor($D$,$U$), in the APPS coding domain, using red/blue-team experiments to assess safety and usefulness. The key finding is that a powerful decomposer $D$ can substantially improve safety over traditional trusted monitoring, primarily via enhanced monitor performance, while overall usefulness can decline in APPS due to task-decomposition costs; Factor($T$,$U$) shows very low usefulness. The results suggest that factored cognition holds promise in more complex tasks, where decomposition may provide safety benefits without sacrificing performance, and highlight the need for further study across settings and decomposition strategies.

Abstract

The field of AI Control seeks to develop robust control protocols, deployment safeguards for untrusted AI which may be intentionally subversive. However, existing protocols that rely on weaker monitors to detect unsafe behavior often fail on complex tasks beyond the monitor's comprehension. We develop control protocols based on factored cognition, in which a trusted model decomposes a task, an untrusted model solves each resultant child task in isolation, and the results are reassembled into a full solution. These protocols may improve safety by several means, such as by simplifying the context for monitors, or by obscuring vulnerabilities in the environment. We implement our protocols in the APPS coding setting and red team them against backdoor attempts from an adversarial GPT-4.1 Nano. We find that: (i) Adding factored cognition to a trusted monitoring protocol can boost safety from 41% to 63%; (ii) Safety improves because monitor performance improves; (iii) Factored cognition makes it no harder for capable LLMs to write backdoors in APPS. While our protocols show low usefulness in APPS, they hold promise for more complex tasks.

Figures (8)

• Figure 1: Functionality of the Factor($T$,$U$) factored cognition control protocol. Given a root task, the decomposer $T$ creates child tasks and reassembly logic. Each child task is implemented by $U$ under trusted monitoring in an isolated context. The reassembly logic is then used to compose the child solutions into a solution to the root task.
• Figure 2: Safety performance of Factor($D$,$U$) protocols with different decomposition granularities relative to trusted monitoring. Arrows indicate the direction correlated with higher safety. The rightmost cluster shows ASR when using o3-mini to find exploits instead of GPT-4.1 Nano.
• Figure 3: (a) Trusted monitor ROC curves for classifying attacks vs. honest policy solutions from each protocol. (b) Attack suspiciousness distributions. (c) $U_H$ suspiciousness distributions. The vertical line segments indicate the distribution means. The area under each curve is proportional to the quantity of suspiciousness scores produced by the protocol over 1000 root tasks.
• Figure 4: Safety metrics when using an o3-mini exploit finder.
• Figure 5: Bootstrap distributions of safety when using an o3-mini exploit finder. Each point shows the safety of each protocol conditional on identical bootstrap samples of root tasks across protocols. This applies to both the usefulness and safety runs.