When Attention Betrays: Erasing Backdoor Attacks in Robotic Policies by Reconstructing Visual Tokens

TL;DR

This work addresses backdoor threats in vision-language-action models used for robotic manipulation by introducing Bera, a test-time backdoor erasure framework. Bera exploits a deep-layer attention grabbing mechanism to locate anomalous image tokens via latent-space localization, applies attention-guided filtering to prune trigger regions, and reconstructs a trigger-free image without retraining the VLA. In extensive real-robot experiments, Bera achieves substantial reductions in attack success rates while preserving nominal task performance and showing strong recovery capabilities across multiple platforms and tasks, outperforming baselines. The approach offers a practical, plug-and-play defense that enhances the safety and reliability of deployed robotic systems in the presence of multimodal backdoors.

Abstract

Downstream fine-tuning of vision-language-action (VLA) models enhances robotics, yet exposes the pipeline to backdoor risks. Attackers can pretrain VLAs on poisoned data to implant backdoors that remain stealthy but can trigger harmful behavior during inference. However, existing defenses either lack mechanistic insight into multimodal backdoors or impose prohibitive computational costs via full-model retraining. To this end, we uncover a deep-layer attention grabbing mechanism: backdoors redirect late-stage attention and form compact embedding clusters near the clean manifold. Leveraging this insight, we introduce Bera, a test-time backdoor erasure framework that detects tokens with anomalous attention via latent-space localization, masks suspicious regions using deep-layer cues, and reconstructs a trigger-free image to break the trigger-unsafe-action mapping while restoring correct behavior. Unlike prior defenses, Bera requires neither retraining of VLAs nor any changes to the training pipeline. Extensive experiments across multiple embodied platforms and tasks show that Bera effectively maintains nominal performance, significantly reduces attack success rates, and consistently restores benign behavior from backdoored outputs, thereby offering a robust and practical defense mechanism for securing robotic systems.

Figures (8)

• Figure 1: Fine-tuning vulnerabilities in robotics. Poisoned dataset can imprint backdoors, causing a pre-trained manipulation policy to exhibit unsafe behaviors after fine-tuning.
• Figure 2: The Bera workflow. Guided by the observation that deep layers reveal stronger trigger-specific attention, we first localize (step ❶) outlying image tokens by contrasting test-time embeddings against a clean reference manifold (Sec. \ref{['sec:localization']}). We then exploit multi-layer attention to prune spurious detections and filter (step ❷) trigger-relevant regions (Sec. \ref{['sec:filtering']}). Finally, a localized masking strategy coupled with an erasure (step ❸) decoder reconstructs a trigger-free view, breaking the trigger-to-action mapping without retraining (Sec. \ref{['sec:erasure']}).
• Figure 3: Visualization of hierarchical attention. In shallow self-attention layers, activation patterns remain largely consistent with those of normal inputs, whereas in deeper self-attention layers, attention is notably grabbed toward trigger-relevant features.
• Figure 4: T-SNE visualization and mechanism of erasing backdoor. (a) T-SNE visualization shows that poisoned image tokens (marked in black) form clusters adjacent to the normal feature distribution, enhancing attack stealth. (b) Our erasure framework disrupts the trigger-to-unsafe-action mapping by masking anomalous features and reconstructing a purified image via the decoder.
• Figure 5: Qualitative case study with Bera. On the Grasping Fanta task using DexGraspVLA, Bera suppresses trigger-induced behaviors and restores the intended grasp without compromising clean performance. Further details are provided in Sec. \ref{['sec: Recovery Performance']}.