On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations

TL;DR

This work systematically evaluates Vision–Language–Action models under 17 multi-modal perturbations and finds action disturbances are particularly harmful, while existing visual-robust methods do not generalize across modalities. It introduces RobustVLA, a joint input/output robustness framework built on the $\pi_0$ backbone that employs worst-case action noise in a flow-matching objective and a UCB-driven perturbation selector for inputs, achieving substantial robustness gains on LIBERO and strong real-world performance with limited demonstrations. The approach yields 12.6% and 10.4% absolute robustness improvements on $\pi_0$ and OpenVLA backbones, respectively, and 50.6x faster inference than visual-robust baselines, with 65.6% gains observed in four-modality real-world tests. These results demonstrate the practicality of multi-modal robustness for VLA models in both simulation and real deployment, offering a scalable path toward more reliable embodied AI systems.

Abstract

In Vision-Language-Action (VLA) models, robustness to real-world perturbations is critical for deployment. Existing methods target simple visual disturbances, overlooking the broader multi-modal perturbations that arise in actions, instructions, environments, and observations. Here, we first evaluate the robustness of mainstream VLAs under 17 perturbations across four modalities. We find (1) actions as the most fragile modality, (2) Existing visual-robust VLA do not gain robustness in other modality, and (3) pi0 demonstrates superior robustness with a diffusion-based action head. To build multi-modal robust VLAs, we propose RobustVLA against perturbations in VLA inputs and outputs. For output robustness, we perform offline robust optimization against worst-case action noise that maximizes mismatch in flow matching objective. This can be seen as adversarial training, label smoothing, and outlier penalization. For input robustness, we enforce consistent actions across input variations that preserve task semantics. To account for multiple perturbations, we formulate robustness as a multi-armed bandit problem and apply an upper confidence bound algorithm to automatically identify the most harmful noise. Experiments on LIBERO demonstrate our RobustVLA delivers absolute gains over baselines of 12.6% on the pi0 backbone and 10.4% on the OpenVLA backbone across all 17 perturbations, achieving 50.6x faster inference than existing visual-robust VLAs, and a 10.4% gain under mixed perturbations. Our RobustVLA is particularly effective on real-world FR5 robot with limited demonstrations, showing absolute gains by 65.6% under perturbations of four modalities.

Figures (6)

• Figure 1: Framework of our paper. We evaluate VLA robustness under 17 uncertainties across 4 modalities. Based on the findings, we enhance robustness against both VLA inputs and outputs.
• Figure 2: Overview of 17 uncertainty types spanning observation, environment, instruction, and action modalities, used in our evaluation of VLA robustness.
• Figure 3: Selected results of robustness evaluation. Numerical results available in Section. \ref{['sec:experiments']}.
• Figure 4: RobustVLA improves robustness on the OpenVLA backbone, achieves fast inference speed, and withstands mixed perturbations.
• Figure 5: Real-world robustness results. Our RobustVLA is highly effective with scarce demonstrations, while baselines fail due to imprecise action control, obscure observation input, OOD observation and language misinterpretations.