Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Value Vision-Language-Action Planning & Search

Ali Salamatian, Ke, Ren, Kieran Pattison, Cyrus Neary

TL;DR

This work tackles the brittleness of Vision–Language–Action policies under distribution shift by augmenting a fixed VLA backbone with a lightweight state-value head, enabling explicit future-return estimates during Monte Carlo Tree Search planning. The value head, trained on Monte Carlo returns from VLA rollouts, is integrated into a VLAPS-style scoring rule to bias search toward high-value regions, improving planning efficiency and success on LIBERO manipulation tasks. Results show a gain of about 5–6 percentage points in spatial/object suites and a reduction of 5–14% in MCTS simulations, demonstrating meaningful gains with a low-cost addition. The approach highlights a practical path to grounding generalist robotic policies with learned value signals, while outlining directions for broader generalization and more data-efficient data collection.

Abstract

Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic manipulation, yet they remain fundamentally limited by their reliance on behavior cloning, leading to brittleness under distribution shift. While augmenting pretrained models with test-time search algorithms like Monte Carlo Tree Search (MCTS) can mitigate these failures, existing formulations rely solely on the VLA prior for guidance, lacking a grounded estimate of expected future return. Consequently, when the prior is inaccurate, the planner can only correct action selection via the exploration term, which requires extensive simulation to become effective. To address this limitation, we introduce Value Vision-Language-Action Planning and Search (V-VLAPS), a framework that augments MCTS with a lightweight, learnable value function. By training a simple multilayer perceptron (MLP) on the latent representations of a fixed VLA backbone (Octo), we provide the search with an explicit success signal that biases action selection toward high-value regions. We evaluate V-VLAPS on the LIBERO robotic manipulation suite, demonstrating that our value-guided search improves success rates by over 5 percentage points while reducing the average number of MCTS simulations by 5-15 percent compared to baselines that rely only on the VLA prior.

Value Vision-Language-Action Planning & Search

TL;DR

This work tackles the brittleness of Vision–Language–Action policies under distribution shift by augmenting a fixed VLA backbone with a lightweight state-value head, enabling explicit future-return estimates during Monte Carlo Tree Search planning. The value head, trained on Monte Carlo returns from VLA rollouts, is integrated into a VLAPS-style scoring rule to bias search toward high-value regions, improving planning efficiency and success on LIBERO manipulation tasks. Results show a gain of about 5–6 percentage points in spatial/object suites and a reduction of 5–14% in MCTS simulations, demonstrating meaningful gains with a low-cost addition. The approach highlights a practical path to grounding generalist robotic policies with learned value signals, while outlining directions for broader generalization and more data-efficient data collection.

Abstract

Vision-Language-Action (VLA) models have emerged as powerful generalist policies for robotic manipulation, yet they remain fundamentally limited by their reliance on behavior cloning, leading to brittleness under distribution shift. While augmenting pretrained models with test-time search algorithms like Monte Carlo Tree Search (MCTS) can mitigate these failures, existing formulations rely solely on the VLA prior for guidance, lacking a grounded estimate of expected future return. Consequently, when the prior is inaccurate, the planner can only correct action selection via the exploration term, which requires extensive simulation to become effective. To address this limitation, we introduce Value Vision-Language-Action Planning and Search (V-VLAPS), a framework that augments MCTS with a lightweight, learnable value function. By training a simple multilayer perceptron (MLP) on the latent representations of a fixed VLA backbone (Octo), we provide the search with an explicit success signal that biases action selection toward high-value regions. We evaluate V-VLAPS on the LIBERO robotic manipulation suite, demonstrating that our value-guided search improves success rates by over 5 percentage points while reducing the average number of MCTS simulations by 5-15 percent compared to baselines that rely only on the VLA prior.
Paper Structure (15 sections, 5 equations, 3 figures, 3 tables)

This paper contains 15 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision–Language–Action Policies
  4. Monte Carlo Tree Search
  5. Learned Value Functions for Tree Search and VLA Planning
  6. Methodology
  7. Data Collection
  8. Value Head Training
  9. Value Integration
  10. Experiments and Results
  11. Dataset and Training Setup
  12. Quantitative Evaluation
  13. Qualitative Analysis
  14. Discussion and Limitations
  15. Conclusions and Future Work

Figures (3)

  • Figure 1: Overview of our value-guided VLAPS extension. At each MCTS node, the current observations and language instruction (e.g., “Move ketchup to basket”) are passed through the frozen VLA backbone and our value head (MLP) to produce a scalar value estimate. This value is attached to the corresponding node and used in the VLAPS scoring rule to bias node selection. Nodes with higher predicted value (green) are selected more often and tend to lead to successful task completions, while low-value nodes (red) are down-weighted during search.
  • Figure 2: t-SNE projection of transformer readouts from LIBERO tasks, colored by value target.
  • Figure 3: Example qualitative evaluation showing the predicted value throughout a successful trajectory.