Improving Robustness of Vision-Language-Action Models by Restoring Corrupted Visual Inputs

TL;DR

This work identifies a critical vulnerability in Vision-Language-Action models: sensitivity to image corruptions that degrade end-to-end task success. It introduces the Corruption Restoration Transformer (CRT), a plug-and-play, upstream restoration module that reconstructs clean observations from corrupted inputs without fine-tuning the VLA. Through adversarial training on paired clean/corrupted frames from LIBERO-10 and Meta-World MT50, CRT leverages SPT, RoPE, and LSA to preserve high-frequency details and robustly invert artifacts. Experimental results show substantial recovery of performance for large VLAs and meaningful improvements for smaller ones, with CRT imposing minimal latency and memory overhead. The modular design enables deployment only when corruption is detected, offering a practical path toward robust real-world VLAs in noisy robotic environments.

Abstract

Vision-Language-Action (VLA) models have emerged as a dominant paradigm for generalist robotic manipulation, unifying perception and control within a single end-to-end architecture. However, despite their success in controlled environments, reliable real-world deployment is severely hindered by their fragility to visual disturbances. While existing literature extensively addresses physical occlusions caused by scene geometry, a critical mode remains largely unexplored: image corruptions. These sensor-level artifacts, ranging from electronic noise and dead pixels to lens contaminants, directly compromise the integrity of the visual signal prior to interpretation. In this work, we quantify this vulnerability, demonstrating that state-of-the-art VLAs such as $π_{0.5}$ and SmolVLA, suffer catastrophic performance degradation, dropping from 90\% success rates to as low as 2\%, under common signal artifacts. To mitigate this, we introduce the Corruption Restoration Transformer (CRT), a plug-and-play and model-agnostic vision transformer designed to immunize VLA models against sensor disturbances. Leveraging an adversarial training objective, CRT restores clean observations from corrupted inputs without requiring computationally expensive fine-tuning of the underlying model. Extensive experiments across the LIBERO and Meta-World benchmarks demonstrate that CRT effectively recovers lost performance, enabling VLAs to maintain near-baseline success rates, even under severe visual corruption.

Figures (4)

• Figure 1: System Pipeline. The CRT module is inserted upstream of the VLA. It intercepts the corrupted simulator observation $x'$, restores it to $\hat{x}$, and passes the clean estimate to the policy network for action prediction.
• Figure 2: CRT Adversarial Training. The CRT Generator ($\mathcal{G}$) receives a corrupted input $x'$ and produces a reconstructed observation $\hat{x}$. The network is optimized via a multi-objective loss function that combines pixel-wise fidelity $\mathcal{L}_{L1}$, structural similarity $\mathcal{L}_{SSIM}$, and adversarial feedback $\mathcal{L}_{adv}$ from the Discriminator $\mathcal{D}$ to ensure effective reconstructions.
• Figure 3: Architecture of Corruption Restoration Transformer (CRT) The model utilizes four specialized mechanisms: 1) Shifted Patch Tokenization (SPT) to recover local spatial dependencies; 2) Rotary Positional Embeddings (RoPE) for robust relative spatial encoding; 3) Locality Self-Attention (LSA) to sharpen textural details. The deep Transformer backbone enables the model to decouple complex corruption artifacts from the underlying semantic scene, finally reshaping the latent tokens into a restored RGB observation.
• Figure 4: Visual Corruption Types. From top-left: (a) Clean Baseline, (b) Centered Square, (c) Gaussian Noise, (d) Horizontal Lines (0.5), (e) Horizontal Lines (0.2), (f) Water Drops.