Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey

Juhee Kim, Xiaoyuan Liu, Zhun Wang, Shi Qiu, Bo Li, Wenbo Guo, Dawn Song

TL;DR

This work introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.

Abstract

AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.

The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey

TL;DR

This work introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.

Abstract

AI agents that combine large language models with non-AI system components are rapidly emerging in real-world applications, offering unprecedented automation and flexibility. However, this unprecedented flexibility introduces complex security challenges fundamentally different from those in traditional software systems. This paper presents the first systematic and comprehensive survey of AI agent security, including an analysis of the design space, attack landscape, and defense mechanisms for secure AI agent systems. We further conduct case studies to point out existing gaps in securing agentic AI systems and identify open challenges in this emerging domain. Our work also introduces the first systematic framework for understanding the security risks and defense strategies of AI agents, serving as a foundation for building both secure agentic systems and advancing research in this critical area.
Paper Structure (35 sections, 4 figures, 4 tables)

This paper contains 35 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Overview
  3. Design Landscape of Agentic AI Systems
  4. Design Components
  5. Design Dimensions and Security Implications
  6. Attack Landscape of Agentic AI Systems
  7. Attack Vectors
  8. Security Risks
  9. System-level Analysis of Agent Risks
  10. Attack Methods
  11. Defense Landscape of Agentic AI Systems
  12. Security Goals
  13. Runtime Protection
  14. Input Guardrail
  15. Output Guardrail
  16. ...and 20 more sections

Figures (4)

  • Figure 1: Overview of an AI Agent's structure.
  • Figure 2: Demonstrations of attack vectors (\ref{['vector:indirectinjection']}-\ref{['vector:memory']}) and security risks (\ref{['risk:attack-surface']}-\ref{['risk:availability']}) against AI agents.
  • Figure 3: Relationship between agent design dimensions and risks, and the connection across different risks.
  • Figure 4: The defense landscape of AI agents. We illustrate key defense mechanisms and where they can be applied within the agent system.