ExpReS-VLA: Specializing Vision-Language-Action Models Through Experience Replay and Retrieval

TL;DR

ExpReS-VLA addresses the adaptation gap for vision-language-action models by specializing pre-trained VLAs to deployment environments while preventing catastrophic forgetting. It achieves this with a compact, on-device experience replay buffer of vision-embedding representations, retrieval-augmented training, and Thresholded Hybrid Contrastive Loss that learns from both successes and failures. The method runs entirely on a single consumer GPU, combining a frozen vision backbone with LoRA-based updates to enable rapid specialization (about 31 seconds with 12 demonstrations). Across LIBERO simulation benchmarks and real-robot experiments, ExpReS-VLA delivers substantial improvements in in-domain and out-of-domain success rates, while achieving dramatic memory efficiency (~97% storage reduction) and practical on-device deployment.

Abstract

Vision-Language-Action models such as OpenVLA show impressive zero-shot generalization across robotic manipulation tasks but often fail to adapt efficiently to new deployment environments. In many real-world applications, consistent high performance on a limited set of tasks is more important than broad generalization. We propose ExpReS-VLA, a method for specializing pre-trained VLA models through experience replay and retrieval while preventing catastrophic forgetting. ExpReS-VLA stores compact feature representations from the frozen vision backbone instead of raw image-action pairs, reducing memory usage by approximately 97 percent. During deployment, relevant past experiences are retrieved using cosine similarity and used to guide adaptation, while prioritized experience replay emphasizes successful trajectories. We also introduce Thresholded Hybrid Contrastive Loss, which enables learning from both successful and failed attempts. On the LIBERO simulation benchmark, ExpReS-VLA improves success rates from 82.6 to 93.1 percent on spatial reasoning tasks and from 61 to 72.3 percent on long-horizon tasks. On physical robot experiments with five manipulation tasks, it reaches 98 percent success on both seen and unseen settings, compared to 84.7 and 32 percent for naive fine-tuning. Adaptation takes 31 seconds using 12 demonstrations on a single RTX 5090 GPU, making the approach practical for real robot deployment.

Figures (5)

• Figure 1: ExpReS-VLA takes in the RGB and prompt input, treats it with e coding, then passes that encoding to the buffer, used for retrivals and prioritizing the learning data for the policy. All of this runs on a single edge device, RTX 5090, and is optimized for performance on the Franka 7-DOF arm.
• Figure 2: ExpReS-VLA system architecture. The frozen encoder extracts embeddings from RGB frames using fused SigLIP and DINOv2 features, which are stored in a compressed replay buffer. During adaptation, the system retrieves the top-k most similar experiences via RAG and combines them with current observations. A prioritizer with error tracking selects high-value samples for mini-batch construction, which are then processed through the unified vector space for gradient updates to the policy $\pi_{\theta}$.
• Figure 3: LIBERO Benchmark task suites used for simulation evaluation. (a) LIBERO-Long: Multi-step tasks requiring sequential manipulation of multiple objects with extended horizon. (b) LIBERO-Object: Object-centric reasoning tasks focusing on identifying and manipulating specific items among distractors. (c) LIBERO-Goal: Goal-directed tasks requiring understanding of target configurations and spatial relationships. (d) LIBERO-Spatial: Spatial reasoning tasks emphasizing precise positioning and geometric constraints. Each suite contains 10 distinct tasks evaluated over 50 roll-outs with 5 random seeds, totaling 2,500 trials per suite. Tasks vary in complexity from simple pick-and-place to multi-stage rearrangements requiring precise sequencing.
• Figure 4: Learning curves showing cumulative success rates across 500 rollouts for each LIBERO task suite. Moving average smoothing (window=10) applied for clarity. The full ExpReS-VLA model (purple) consistently outperforms ablations, with particularly strong gains when all components are combined.
• Figure 5: Physical robot evaluation tasks. We evaluate ExpReS-VLA on five manipulation tasks using a 7-DOF Franka Emika Panda arm: (a) Place white mug in bowl: Precise placement requiring 5cm accuracy, (b) Stack all bowls: Multi-object manipulation with stability constraints, (c) Push grey bowl: Contact-rich pushing toward glass target, (d) Knock pringles can: Dynamic motion to topple can, (e) Move cans: Rearrangement of 7UP next to Pepsi can. Each task is evaluated with 30 in-distribution trials matching training conditions and 10 out-of-distribution trials with variations including different backgrounds (white/patterned tablecloths), unseen objects (different colored bowls, Diet 7UP), and novel initial configurations.