Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving

TL;DR

This work addresses the vulnerability of autonomous-driving vision-language systems to black-box adversarial attacks. It introduces Cascading Adversarial Disruption (CAD), combining Decision Chain Disruption and Risky Scene Induction to perturb both low-level reasoning and high-level safety assessments, optimized via a multi-term objective. Empirical results show CAD achieves state-of-the-art attack effectiveness across diverse AD and general VLMs in digital and real-world tests, accompanied by the CADA dataset to foster further research. The study also explores countermeasures, ablations, and a dataset tailored to scene- and object-level perturbations, highlighting the practical risks and need for robust defenses in safety-critical settings.

Abstract

Vision-language models (VLMs) have significantly advanced autonomous driving (AD) by enhancing reasoning capabilities; however, these models remain highly susceptible to adversarial attacks. While existing research has explored white-box attacks to some extent, the more practical and challenging black-box scenarios remain largely underexplored due to their inherent difficulty. In this paper, we take the first step toward designing black-box adversarial attacks specifically targeting VLMs in AD. We identify two key challenges for achieving effective black-box attacks in this context: the effectiveness across driving reasoning chains in AD systems and the dynamic nature of driving scenarios. To address this, we propose Cascading Adversarial Disruption (CAD). It first introduces Decision Chain Disruption, which targets low-level reasoning breakdown by generating and injecting deceptive semantics, ensuring the perturbations remain effective across the entire decision-making chain. Building on this, we present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios that are likely to result in critical errors in the current driving contexts. Extensive experiments conducted on multiple AD VLMs and benchmarks demonstrate that CAD achieves state-of-the-art attack effectiveness, significantly outperforming existing methods (+13.43% on average). Moreover, we validate its practical applicability through real-world attacks on AD vehicles powered by VLMs, where the route completion rate drops by 61.11% and the vehicle crashes directly into the obstacle vehicle with adversarial patches. Finally, we release CADA dataset, comprising 18,808 adversarial visual-question-answer pairs, to facilitate further evaluation and research in this critical domain. Our codes and dataset will be available after paper's acceptance.

Figures (9)

• Figure 1: Illustration of our CAD black-box attack. The added visual perturbations cause the model to misinterpret a red traffic light as green. This misjudgment leads to incorrect driving actions, ultimately resulting in a collision.
• Figure 2: Attack Framework. Our approach introduces Decision Chain Disruption, which targets low-level reasoning breakdown by generating and injecting deceptive semantics. Building on this, we present Risky Scene Induction, which addresses dynamic adaptation by leveraging a surrogate VLM to understand and construct high-level risky scenarios.
• Figure 3: Experimental results of LMDrive in the closed-loop simulator, comparing the driving operations of LMDrive at the same location on the same route with and without attack.
• Figure 4: Overview of the VLM-driven pipeline. The system adopts a client-server architecture, using Dolphins to generate high-level commands and LLaMA to translate them into low-level control instructions.
• Figure 5: Illustration of experimental scenarios and adversarial patches for Jetbot and LIMO in real-world settings.