Observing and Controlling Features in Vision-Language-Action Models

TL;DR

This work demonstrates on different VLA architectures that VLAs possess interpretable internal structure amenable to online adaptation without fine-tuning, enabling real-time alignment with user preferences and task requirements.

Abstract

Vision-Language-Action Models (VLAs) have shown remarkable progress towards embodied intelligence. While their architecture partially resembles that of Large Language Models (LLMs), VLAs exhibit higher complexity due to their multi-modal inputs/outputs and often hybrid nature of transformer and diffusion heads. This is part of the reason why insights from mechanistic interpretability in LLMs, which explain how the internal model representations relate to their output behavior, do not trivially transfer to VLA counterparts. In this work, we propose to close this gap by introducing and analyzing two main concepts: feature-observability and feature-controllability. In particular, we first study features that are linearly encoded in representation space, and show how they can be observed by means of a linear classifier. Then, we use a minimal linear intervention grounded in optimal control to accurately place internal representations and steer the VLA's output towards a desired region. Our results show that targeted, lightweight interventions can reliably steer a robot's behavior while preserving closed-loop capabilities. We demonstrate on different VLA architectures ($π_{0.5}$ and OpenVLA) through simulation experiments that VLAs possess interpretable internal structure amenable to online adaptation without fine-tuning, enabling real-time alignment with user preferences and task requirements.

Figures (10)

• Figure 1: We present the combination of feature-observability and feature-controllability: a framework to observe and control robot behavior through accessing and modifying internal VLA representations. A linear observer extracts features from the Transformer's internal representations (left), while a minimal linear controller steers these representations to align outputs with desired constraints (right), enabling real-time policy steering without fine-tuning.
• Figure 2: Schematic representation of two VLA architectures. On the left, a Transformer-based VLA features an autorregressive Transformer architecture. On the right, a Transformer-Flow-Matching hybrid VLA is composed of a Transformer architecture and a Flow Matching block (referred to as the "action expert"), where the Flow Matching layers attend to their paired counterparts in the Transformer.
• Figure 3: On the top, results from training a linear classifier via Algorithm \ref{['alg:offline']} on the $\pi_{0.5}$ transformer layers on test data from Libero dataset. Left: the maximum absolute error (MAE) of the trained classifier compared with the MAE from mean training prediction, used as a baseline. Right: accuracy of the trained classifier compared with accuracy from majority class prediction, used as a baseline. On the bottom, identical representation, using OpenVLA model on the BridgeData V2 dataset. In all cases, only results for the best performant layer are displayed.
• Figure 4: Effect of linear interventions applied to the representation space of $\pi_{0.5}$ at different layers, using episodes from Libero dataset. On the top, mean change in the delta yaw action, averaged across episodes, as a function of layer where the perturbation (linear offset added to the representation) is applied. The middle similarly shows the mean change in delta gripper action. The perturbation is added to the representation at a single layer $\ell$. Each curve corresponds to a different strength $\alpha$ of the perturbation. In the bottom, the $L_2$-norm of the representation vector as a function of layer depth is included, showing that representation magnitude increases with depth, which explains the diminishing effect of a fixed perturbation in deeper layers.
• Figure 5: Visualization of the representation projected into the classifier image space under different interventions of $\pi_{0.5}$'s and OpenVLA's transformer layer 9. The proposed minimal controller from equation \ref{['eq:u_closed_form']}, observed via equation \ref{['eqn:observer']}, constrains the image of the intervened representation to lie within the target bounds $[\zeta_{\min}, \zeta_{\max}]$, while other interventions as well as un-intervened representations fall outside.

Theorems & Definitions (4)

• Definition 1: Feature-Observability
• Definition 2: Feature-Controllability
• Remark 1
• Remark 2