Cognitive Control Architecture (CCA): A Lifecycle Supervision Framework for Robustly Aligned AI Agents

TL;DR

The paper tackles the vulnerability of autonomous LLM agents to Indirect Prompt Injection by proposing the Cognitive Control Architecture (CCA), a two-layer framework for full-lifecycle cognitive supervision. It combines a proactive Intent Graph that enforces control-flow and data-flow integrity with a reactive Tiered Adjudicator that performs high-fidelity, multi-dimensional deviation assessment via an Intent Alignment Score. Key contributions include the formalization of the Intent Graph, the design of the multi-component S_align metric (including S_sem, S_causal, S_prov, S_risk), and an extensive evaluation on the AgentDojo benchmark showing superior security with minimal impact on functionality and improved efficiency. The results demonstrate that CCA effectively mitigates sophisticated IPI attacks while maintaining high task performance, offering a scalable blueprint for safer autonomous agents in real-world settings.

Abstract

Autonomous Large Language Model (LLM) agents exhibit significant vulnerability to Indirect Prompt Injection (IPI) attacks. These attacks hijack agent behavior by polluting external information sources, exploiting fundamental trade-offs between security and functionality in existing defense mechanisms. This leads to malicious and unauthorized tool invocations, diverting agents from their original objectives. The success of complex IPIs reveals a deeper systemic fragility: while current defenses demonstrate some effectiveness, most defense architectures are inherently fragmented. Consequently, they fail to provide full integrity assurance across the entire task execution pipeline, forcing unacceptable multi-dimensional compromises among security, functionality, and efficiency. Our method is predicated on a core insight: no matter how subtle an IPI attack, its pursuit of a malicious objective will ultimately manifest as a detectable deviation in the action trajectory, distinct from the expected legitimate plan. Based on this, we propose the Cognitive Control Architecture (CCA), a holistic framework achieving full-lifecycle cognitive supervision. CCA constructs an efficient, dual-layered defense system through two synergistic pillars: (i) proactive and preemptive control-flow and data-flow integrity enforcement via a pre-generated "Intent Graph"; and (ii) an innovative "Tiered Adjudicator" that, upon deviation detection, initiates deep reasoning based on multi-dimensional scoring, specifically designed to counter complex conditional attacks. Experiments on the AgentDojo benchmark substantiate that CCA not only effectively withstands sophisticated attacks that challenge other advanced defense methods but also achieves uncompromised security with notable efficiency and robustness, thereby reconciling the aforementioned multi-dimensional trade-off.

Figures (5)

• Figure 1: An illustrative example of a multi-step Indirect Prompt Injection (IPI) attack. The agent begins with a benign user goal (summarizing meeting notes). However, its read_document tool ingests malicious instructions embedded within the external document. This contamination hijacks the agent's behavior, causing it to deviate from the original task and execute an unauthorized send_email action, ultimately leading to data exfiltration.
• Figure 2: The Cognitive Control Architecture (CCA). The CCA operates in two layers. First Layer (Pillar I): Upon receiving a user goal, the agent proactively generates an Intent Graph, which defines the legitimate sequence of tool calls. Execution: In the execution loop, each proposed action is checked against the Intent Graph. Compliant actions are executed directly. Second Layer (Pillar II): Any action that deviates from the graph triggers the Tiered Adjudicator. This adjudicator calculates a multi-faceted Intent Alignment Score ($S_{align}$), composed of semantic, causal, trust, and risk assessments, to make a final decision: approve the action, block it, or ask the user for clarification.
• Figure 3: Boxplot distributions of the Intent Alignment Score $S_{\text{align}}$ for the full model and four ablation settings, evaluated on all deviating actions.
• Figure 4: CCA outperforms baselines in security, functionality, and efficiency. (a) DSR--TSR trade-off, where $\mathrm{DSR} = 100\% - \mathrm{ASR}$ and $\mathrm{TSR} = \mathrm{UA}$ (higher is better on both axes). (b,c) Average execution tokens per scenario. (d) Average planning tokens for the Intent Graph in each agent scenario.
• Figure 5: Schematics for the different types of IPI attacks evaluated in our work.