Inject Once Survive Later: Backdooring Vision-Language-Action Models to Persist Through Downstream Fine-tuning

TL;DR

INjection into Fine-tUne-inSensitive modulEs (INjection into Fine-tune-inSensitive modulEs), the first backdoor attack framework for VLA base models that remains effective even with arbitrary user fine-tuning, uncovered a critical threat: backdoors implanted before distribution can persist through fine-tuning and remain effective at deployment.

Abstract

Vision-Language-Action (VLA) models have become foundational to modern embodied AI systems. By integrating visual perception, language understanding, and action planning, they enable general-purpose task execution across diverse environments. Despite their importance, the security of VLA models remains underexplored -- particularly in the context of backdoor attacks, which pose realistic threats in physical-world deployments. While recent methods attempt to inject backdoors into VLA models, these backdoors are easily erased during downstream adaptation, as user-side fine-tuning with clean data significantly alters model parameters, rendering them impractical for real-world applications. To address these challenges, we propose INFUSE (INjection into Fine-tUne-inSensitive modulEs), the first backdoor attack framework for VLA base models that remains effective even with arbitrary user fine-tuning. INFUSE begins by analyzing parameter sensitivity across diverse fine-tuning scenarios to identify modules that remain largely unchanged -- the fine-tune-insensitive modules. It then injects backdoors into these stable modules while freezing the rest, ensuring malicious behavior persists after extensive user fine-tuning. Comprehensive experiments across multiple VLA architectures demonstrate INFUSE's effectiveness. After user-side fine-tuning, INFUSE maintains mean attack success rates of 91.0% on simulation environments and 79.8% on real-world robot tasks, substantially surpassing BadVLA (38.8% and 36.6%, respectively), while preserving clean-task performance comparable to standard models. These results uncover a critical threat: backdoors implanted before distribution can persist through fine-tuning and remain effective at deployment.

Figures (6)

• Figure 1: Attack persistence comparison before and after user-side clean fine-tuning on four LIBERO tasks. Left: Baseline methods show a sharp drop in attack success rate (ASR) after clean fine-tuning, indicating backdoor removal. Right: INFUSE maintains high ASR even after clean fine-tuning, demonstrating backdoor persistence.
• Figure 2: INFUSE pipeline overview. INFUSE consists of three stages: (1) Fine-tune-Insensitive Module Identification: We analyze parameter changes after fine-tuning the base VLA model on multiple clean environments to identify modules that remain stable (fine-tune-insensitive) and suitable for persistent backdoor injection. (2) Selective Backdoor Injection on Fine-tune-Insensitive Modules: We construct a poisoned dataset with triggers and malicious target actions, then selectively fine-tune only the fine-tune-insensitive modules while freezing the sensitive ones, producing a poisoned base VLA model. (3) User-side Finetuning: We simulate realistic user adaptation by fine-tuning the poisoned base model with clean datasets from different environments, demonstrating that the injected backdoor remains effective even after user-side customization.
• Figure 3: Module sensitivity on OpenVLA-OFT. Log-scale bars report the mean absolute difference, Fisher-normalized difference, and CKA-based activation shift between pre- and post-fine-tuning, aggregated over downstream adaptations spanning Spatial, Goal, Object, LIBERO-10, and real-world trajectories. Panel (d) shows the normalized overall sensitivity score $S_i$. Lower scores indicate fine-tune-insensitive modules that we target for selective injection in Stage 2.
• Figure 4: Attention heatmap comparison before and after fine-tuning. While baseline models lose focus on the trigger after fine-tuning, INFUSE maintains strong attention, indicating persistent backdoor behavior.
• Figure 5: Trajectory comparison of the Normal Model, Poisoned Base Model, and User-Finetuned Poisoned Model. INFUSE retains trajectory deviation after fine-tuning, indicating persistent backdoor behavior.