MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents

TL;DR

The paper addresses vulnerabilities of autonomous driving perception to physical adversarial patches and reframes patch generation as a context-aware, one-shot task. It introduces MAGIC, a collaborative multi-modal LLM agent framework with GAgent, DAgent, and EAgent that jointly generate, deploy, and refine patches within a scene, using self-examination to ensure attack effectiveness and naturalness. Through digital and physical experiments on multiple detectors, MAGIC demonstrates superior attack performance and context-consistent deployment compared with diffusion-based baselines, while also analyzing robustness to real-world factors. The work advances safety research in autonomous driving by integrating scene context into patch generation and deployment, and it discusses ethical considerations, limitations, and directions for defense and broader impact.

Abstract

Physical adversarial attacks in driving scenarios can expose critical vulnerabilities in visual perception models. However, developing such attacks remains challenging due to diverse real-world environments and the requirement for maintaining visual naturality. Building upon this challenge, we reformulate physical adversarial attacks as a one-shot patch generation problem. Our approach generates adversarial patches through a deep generative model that considers the specific scene context, enabling direct physical deployment in matching environments. The primary challenge lies in simultaneously achieving two objectives: generating adversarial patches that effectively mislead object detection systems while determining contextually appropriate deployment within the scene. We propose MAGIC (Mastering Physical Adversarial Generation In Context), a novel framework powered by multi-modal LLM agents to address these challenges. MAGIC automatically understands scene context and generates adversarial patch through the synergistic interaction of language and vision capabilities. In particular, MAGIC orchestrates three specialized LLM agents: The adv-patch generation agent (GAgent) masters the creation of deceptive patches through strategic prompt engineering for text-to-image models. The adv-patch deployment agent (DAgent) ensures contextual coherence by determining optimal deployment strategies based on scene understanding. The self-examination agent (EAgent) completes this trilogy by providing critical oversight and iterative refinement of both processes. We validate our method on both digital and physical levels, i.e., nuImage and manually captured real-world scenes, where both statistical and visual results prove that our MAGIC is powerful and effective for attacking widely applied object detection systems, i.e., YOLO and DETR series.

Figures (10)

• Figure 1: Detection error ratios (i.e., $\epsilon^\mathbf{I}_i$ and $\epsilon^\mathbb{I}_i$) across two image sets (i.e.$\{{\mathbb{I}}_i^{k}\}{k=1}^K$ and $\{{\mathbf{I}}_i^k\}{k=1}^K$) for 15 prompt types ($i={1,\ldots,15}$). The y-axis labels indicate removed features: Shape (S), Color (C), Text (T), and Pattern (P). Patch generated by (a) Stable Diffusion and (b) Midjourney.
• Figure 1: Digital-to-physical transfer visualization results of NDDA and MAGIC.
• Figure 2: Visualization of three generated patches ($\mathbf{P}_1$,$\mathbf{P}_2$,$\mathbf{P}_3$), three environments ($\mathbf{E}_1$,$\mathbf{E}_2$,$\mathbf{E}_3$), and the detection results with YOLOv5 across them. The confidence score (conf.) $>0.5$ means the patch realizes the attack goal. Bar plot at the top: detection error ratios before (blur) and after (red) deployment over all the NDDA patches for each of the environments.
• Figure 2: Viewing distance and angle results for MAGIC patches.
• Figure 3: (a) - (b): candidate patch locations and scales. (c) - (f): the detection confidence across all locations and scales for effective pairs in Fig. \ref{['fig:rq2']}.