Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sci-VLA: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments

Yiwen Pang, Bo Zhou, Changjin Li, Xuanhao Wang, Shengxiang Xu, Deng-Bao Wang, Min-Ling Zhang, Shimin Di

TL;DR

This work tackles the state-gap problem in vision-language-action (VLA) models when executing long-horizon, composite scientific tasks. It introduces Sci-VLA, an LLM-based inference plugin that generates transitional action code to bridge transitions between atomic tasks, enabling reliable long-horizon workflows without retraining. The approach is validated in a digital twin (Autobio MuJoCo) with UR5e, showing a notable 42% average improvement in atomic-task success and enhanced execution coherence, with demonstrated transfer to real laboratories. By integrating transition inference and code insertion, Sci-VLA offers data-efficient, computation-efficient open-ended robotic laboratory capabilities. The work highlights the potential and challenges of sim-to-real transfer and calls for improved action precision and dataset balance to fully realize long-horizon robotic experimentation.

Abstract

Robotic laboratories play a critical role in autonomous scientific discovery by enabling scalable, continuous experimental execution. Recent vision-language-action (VLA) models offer a promising foundation for robotic laboratories. However, scientific experiments typically involve long-horizon tasks composed of multiple atomic tasks, posing a fundamental challenge to existing VLA models. While VLA models fine-tuned for scientific tasks can reliably execute atomic experimental actions seen during training, they often fail to perform composite tasks formed by reordering and composing these known atomic actions. This limitation arises from a distributional mismatch between training-time atomic tasks and inference-time composite tasks, which prevents VLA models from executing necessary transitional operations between atomic tasks. To address this challenge, we propose an Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments. It introduces an LLM-based agentic inference mechanism that intervenes when executing sequential manipulation tasks. By performing explicit transition inference and generating transitional robotic action code, the proposed plugin guides VLA models through missing transitional steps, enabling reliable execution of composite scientific workflows without any additional training. This inference-only intervention makes our method computationally efficient, data-efficient, and well-suited for open-ended and long-horizon robotic laboratory tasks. We build 3D assets of scientific instruments and common scientific operating scenes within an existing simulation environment. In these scenes, we have verified that our method increases the average success rate per atomic task by 42\% during inference. Furthermore, we show that our method can be easily transferred from the simulation to real scientific laboratories.

Sci-VLA: Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments

TL;DR

This work tackles the state-gap problem in vision-language-action (VLA) models when executing long-horizon, composite scientific tasks. It introduces Sci-VLA, an LLM-based inference plugin that generates transitional action code to bridge transitions between atomic tasks, enabling reliable long-horizon workflows without retraining. The approach is validated in a digital twin (Autobio MuJoCo) with UR5e, showing a notable 42% average improvement in atomic-task success and enhanced execution coherence, with demonstrated transfer to real laboratories. By integrating transition inference and code insertion, Sci-VLA offers data-efficient, computation-efficient open-ended robotic laboratory capabilities. The work highlights the potential and challenges of sim-to-real transfer and calls for improved action precision and dataset balance to fully realize long-horizon robotic experimentation.

Abstract

Robotic laboratories play a critical role in autonomous scientific discovery by enabling scalable, continuous experimental execution. Recent vision-language-action (VLA) models offer a promising foundation for robotic laboratories. However, scientific experiments typically involve long-horizon tasks composed of multiple atomic tasks, posing a fundamental challenge to existing VLA models. While VLA models fine-tuned for scientific tasks can reliably execute atomic experimental actions seen during training, they often fail to perform composite tasks formed by reordering and composing these known atomic actions. This limitation arises from a distributional mismatch between training-time atomic tasks and inference-time composite tasks, which prevents VLA models from executing necessary transitional operations between atomic tasks. To address this challenge, we propose an Agentic VLA Inference Plugin for Long-Horizon Tasks in Scientific Experiments. It introduces an LLM-based agentic inference mechanism that intervenes when executing sequential manipulation tasks. By performing explicit transition inference and generating transitional robotic action code, the proposed plugin guides VLA models through missing transitional steps, enabling reliable execution of composite scientific workflows without any additional training. This inference-only intervention makes our method computationally efficient, data-efficient, and well-suited for open-ended and long-horizon robotic laboratory tasks. We build 3D assets of scientific instruments and common scientific operating scenes within an existing simulation environment. In these scenes, we have verified that our method increases the average success rate per atomic task by 42\% during inference. Furthermore, we show that our method can be easily transferred from the simulation to real scientific laboratories.
Paper Structure (23 sections, 5 equations, 12 figures, 2 tables, 1 algorithm)

This paper contains 23 sections, 5 equations, 12 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Robotic Laboratories
  4. Robot Manipulation
  5. Sci-VLA: Agentic VLA Inference Plugin
  6. Finetuning VLA for Atomic Tasks
  7. Inference Atomic Tasks Sequence
  8. Digital Twin
  9. Experiments
  10. Experiments Setup
  11. Results
  12. Basic Validation
  13. Atomic Task Coherence
  14. Long Experimental Operation Tasks
  15. Real Environment Demonstration
  16. ...and 8 more sections

Figures (12)

  • Figure 1: Given a VLA model fine-tuned for atomic tasks in scientific experiments, it may fail to complete the composite task due to the state gap issue. In this paper, we propose a LLM-based agentic inference plugin to generate transitional robotic execution code, enabling execution of the long-horizon tasks.
  • Figure 2: The illustration of the Sci-VLA pipeline. Includes two modules: transitional action generation and transitional action insertion. The plugin and VLA will alternate in executing the sequence of atomic tasks.
  • Figure 3: Some scientific simulated objects we used and their corresponding real object. Top: 3D simulated assets. Bottom: real objects. (a) Pipette. (b) Pipette rack. (c) Centrifuge tube rack. (d) Centrifuge tube plate. (e) 3D printer.
  • Figure 4: Some newly added instrument 3D assets examples. (a) Ozone cleaner. (b) Spin coater. (c) Hot plate.
  • Figure 5: Action trajectory recording diagram for base model $\pi_0$ executing task sequence in 2 situations: simulation and real-world. The yellow lines represent the trajectory of the end effector. The dashed box indicates the trajectory of the transitional actions. Simulated cleaning table task: pick 3 objects into the basket.
  • ...and 7 more figures