Physical Backdoor Attack can Jeopardize Driving with Vision-Large-Language Models

TL;DR

This work exposes a tangible security risk in autonomous driving by introducing BadVLMDriver, the first physical backdoor attack targeting Vision-Large-Language Models. The authors propose an automated, instruction-guided pipeline that (i) embeds physical triggers into scenes using diffusion-based editing and (ii) rewrites model responses with target malicious behaviors via LLM-assisted response modification, followed by replay-based visual instruction tuning to solidify the trigger-behavior mapping. Experiments across two VLMs, five physical triggers, and two dangerous behaviors demonstrate high attack success rates (e.g., up to 92% in a red-balloon scenario) while preserving normal task performance on standard benchmarks, underscoring a critical safety threat. The results emphasize the need for robust defenses against physical backdoors in VLM-enabled autonomous driving and motivate future work on defense strategies and safer deployment guidelines.

Abstract

Vision-Large-Language-models(VLMs) have great application prospects in autonomous driving. Despite the ability of VLMs to comprehend and make decisions in complex scenarios, their integration into safety-critical autonomous driving systems poses serious security risks. In this paper, we propose BadVLMDriver, the first backdoor attack against VLMs for autonomous driving that can be launched in practice using physical objects. Unlike existing backdoor attacks against VLMs that rely on digital modifications, BadVLMDriver uses common physical items, such as a red balloon, to induce unsafe actions like sudden acceleration, highlighting a significant real-world threat to autonomous vehicle safety. To execute BadVLMDriver, we develop an automated pipeline utilizing natural language instructions to generate backdoor training samples with embedded malicious behaviors. This approach allows for flexible trigger and behavior selection, enhancing the stealth and practicality of the attack in diverse scenarios. We conduct extensive experiments to evaluate BadVLMDriver for two representative VLMs, five different trigger objects, and two types of malicious backdoor behaviors. BadVLMDriver achieves a 92% attack success rate in inducing a sudden acceleration when coming across a pedestrian holding a red balloon. Thus, BadVLMDriver not only demonstrates a critical security risk but also emphasizes the urgent need for developing robust defense mechanisms to protect against such vulnerabilities in autonomous driving technologies.

Figures (13)

• Figure 1: Illustration of the security risk of an autonomous vehicle controlled by a VLM. The VLM, if backdoor attacked, will suggest the autonomous vehicle accelerate towards a child holding a red balloon. Images are created with the assistance of DALL·E 3 betker2023improving.
• Figure 2: Illustration of the automated pipeline for BadVLMDriver. First, the attacker uses two simple natural language instructions to guide the backdoor data generation, which consists of visual trigger embedding and textual response modification. Then, with the backdoor and benign samples, the VLM is optimized via visual instruction tuning based on a blending optimization objective. Finally, autonomous driving empowered by the backdoored VLM will behave dangerously in the real world whenever the trigger object appears in the scene.
• Figure 3: Examples of instruction-guided visual trigger embedding.
• Figure 4: The left shows the instruction for obtaining the driving actions from the VLM. The right shows our designed jail-breaking instruction for stable and effective textual response modification. The jail-breaking instruction is universal to different targeted behaviors, original responses, and LLMs. The last sentence can effectively inform the LLM to 'forget' about safety, therefore exactly following our modification instruction.
• Figure 5: Visualization of real-world physical attack. Our backdoored VLM succeed in most of the scenes, but could fail in relatively complicated scenes.