Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Systems Security Foundations for Agentic Computing

Mihai Christodorescu, Earlence Fernandes, Ashish Hooda, Somesh Jha, Johann Rehberger, Khawaja Shams

TL;DR

The paper addresses securing AI agents that operate within complex computer systems, arguing that end-to-end security is needed rather than focusing solely on hardening individual models. It adopts classic security principles and highlights challenges posed by probabilistic TCBs, dynamic task-specific policies, fuzzy boundaries, and prompt injections, supported by 11 real-world attacks. It outlines concrete defenses and a two-pronged open problem space: practical mechanisms for separating instructions from data, access control, and information-flow control, plus long-term questions about probabilistic guarantees and security-aware architectures. The authors aim to guide AI/ML practitioners and security researchers toward a foundation for secure agentic computing through cross-disciplinary collaboration and focused research agendas.

Abstract

This paper articulates short- and long-term research problems in AI agent security and privacy, using the lens of computer systems security. This approach examines end-to-end security properties of entire systems, rather than AI models in isolation. While we recognize that hardening a single model is useful, it is important to realize that it is often insufficient. By way of an analogy, creating a model that is always helpful and harmless is akin to creating software that is always helpful and harmless. The collective experience of decades of cybersecurity research and practice shows that this is insufficient. Rather, constructing an informed and realistic attacker model before building a system, applying hard-earned lessons from software security, and continuous improvement of security posture is a tried-and-tested approach to securing real computer systems. A key goal is to examine where research challenges arise when applying traditional security principles in the context of AI agents. A secondary goal of this report is to distill these ideas for AI and ML practitioners and researchers. We discuss the challenges of applying security principles to agentic computing, present 11 case studies of real attacks on agentic systems, and define a series of new research problems specific to the security of agentic systems.

Systems Security Foundations for Agentic Computing

TL;DR

The paper addresses securing AI agents that operate within complex computer systems, arguing that end-to-end security is needed rather than focusing solely on hardening individual models. It adopts classic security principles and highlights challenges posed by probabilistic TCBs, dynamic task-specific policies, fuzzy boundaries, and prompt injections, supported by 11 real-world attacks. It outlines concrete defenses and a two-pronged open problem space: practical mechanisms for separating instructions from data, access control, and information-flow control, plus long-term questions about probabilistic guarantees and security-aware architectures. The authors aim to guide AI/ML practitioners and security researchers toward a foundation for secure agentic computing through cross-disciplinary collaboration and focused research agendas.

Abstract

This paper articulates short- and long-term research problems in AI agent security and privacy, using the lens of computer systems security. This approach examines end-to-end security properties of entire systems, rather than AI models in isolation. While we recognize that hardening a single model is useful, it is important to realize that it is often insufficient. By way of an analogy, creating a model that is always helpful and harmless is akin to creating software that is always helpful and harmless. The collective experience of decades of cybersecurity research and practice shows that this is insufficient. Rather, constructing an informed and realistic attacker model before building a system, applying hard-earned lessons from software security, and continuous improvement of security posture is a tried-and-tested approach to securing real computer systems. A key goal is to examine where research challenges arise when applying traditional security principles in the context of AI agents. A secondary goal of this report is to distill these ideas for AI and ML practitioners and researchers. We discuss the challenges of applying security principles to agentic computing, present 11 case studies of real attacks on agentic systems, and define a series of new research problems specific to the security of agentic systems.

Paper Structure

This paper contains 15 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Security Principles and Mechanisms
  3. Challenges in Applying Security Engineering to Agentic Computing
  4. The TCB Is Probabilistic
  5. The Security Policy Is Dynamic and Task-Specific
  6. The Security Boundary Is Fuzzy
  7. Prompt Injection $\cong$ Dynamic Code Loading
  8. Case Studies of Real Attacks on Agentic Systems
  9. Open Research Problems
  10. Mechanisms for Separating Instructions and Data
  11. Mechanisms for Access Control and Least Privilege
  12. Mechanisms for Information-Flow Control
  13. Long-Term: Security Guarantees from Probabilistic TCBs
  14. Long-Term: Security-Aware Model Architectures
  15. Conclusion

Figures (3)

  • Figure 1: Standard security architecture showing trusted components in blue and untrusted ones in red.
  • Figure 2: OpenAI Operator Exploit Flow: Starts from an issue on a GitHub project and ends with PII being leaked to the attacker's domain.
  • Figure 3: CVE-2025-55284: Claude Code Data Exfiltration via DNS Lookup (Image Credit: Johann Rehberger/EmbraceTheRed).