Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SaiVLA-0: Cerebrum--Pons--Cerebellum Tripartite Architecture for Compute-Aware Vision-Language-Action

Xiang Shi, Wenlong Huang, Menglin Zou, Xinhai Sun

TL;DR

W wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view.

Abstract

We revisit Vision-Language-Action through a neuroscience-inspired triad. Biologically, the Cerebrum provides stable high-level multimodal priors and remains frozen; the Pons Adapter integrates these cortical features with real-time proprioceptive inputs and compiles intent into execution-ready tokens; and the Cerebellum (ParaCAT) performs fast, parallel categorical decoding for online control, with hysteresis/EMA/temperature/entropy for stability. A fixed-ratio schedule and two-stage feature caching make the system compute-aware and reproducible. Inspired by active, foveated vision, our wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view. The design is modular: upgrading the Cerebrum only retrains the Pons; changing robots only trains the Cerebellum; cerebellum-only RL can further refine control without touching high-level semantics. As a concept-and-protocol paper with preliminary evidence, we outline a timing protocol under matched conditions (GPU, resolution, batch) to verify anticipated efficiency gains. We also report preliminary LIBERO evidence showing that split feature caching reduces training time (7.5h to 4.5h) and improves average success (86.5% to 92.5%) under official N1.5 head-only training, and that SaiVLA0 reaches 99.0% mean success.

SaiVLA-0: Cerebrum--Pons--Cerebellum Tripartite Architecture for Compute-Aware Vision-Language-Action

TL;DR

W wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view.

Abstract

We revisit Vision-Language-Action through a neuroscience-inspired triad. Biologically, the Cerebrum provides stable high-level multimodal priors and remains frozen; the Pons Adapter integrates these cortical features with real-time proprioceptive inputs and compiles intent into execution-ready tokens; and the Cerebellum (ParaCAT) performs fast, parallel categorical decoding for online control, with hysteresis/EMA/temperature/entropy for stability. A fixed-ratio schedule and two-stage feature caching make the system compute-aware and reproducible. Inspired by active, foveated vision, our wrist ROIs are geometrically tied to the end-effector via calibrated projection, providing a movement-stabilized, high-resolution view that is sensitive to fine-grained pose changes and complements the global context of the main view. The design is modular: upgrading the Cerebrum only retrains the Pons; changing robots only trains the Cerebellum; cerebellum-only RL can further refine control without touching high-level semantics. As a concept-and-protocol paper with preliminary evidence, we outline a timing protocol under matched conditions (GPU, resolution, batch) to verify anticipated efficiency gains. We also report preliminary LIBERO evidence showing that split feature caching reduces training time (7.5h to 4.5h) and improves average success (86.5% to 92.5%) under official N1.5 head-only training, and that SaiVLA0 reaches 99.0% mean success.
Paper Structure (54 sections, 10 equations, 3 figures, 7 tables)

This paper contains 54 sections, 10 equations, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. VLA and Robot Foundation Models.
  4. Freezing, Adapters and Representation Alignment.
  5. Action Representations and Decoders.
  6. Multi-layer Feature Fusion.
  7. Hierarchical & Tripartite Control.
  8. Structured Prompts & ROI.
  9. Indirect evidence from prior art.
  10. Method
  11. Overall Architecture (Tripartite).
  12. Design hypotheses (testable predictions).
  13. Cerebrum (Frozen VLM).
  14. Pons Adapter.
  15. Cerebellum. ParaCAT (Parallel Categorical Action Transformer; $K\times D$ queries).
  16. ...and 39 more sections

Figures (3)

  • Figure 1: Overview: Frozen Cerebrum emits context tokens sparsely; the Cerebellum fuses image+text+state+brain tokens at high frequency and outputs categorical deltas.
  • Figure 2: ROI projection pipeline. End-effector poses are projected into image coordinates via calibrated intrinsics/extrinsics to produce geometry-tied wrist crops; ROIs complement the main view with movement-stabilized, high-resolution contact cues.
  • Figure 3: Real-robot simple grasping: grasping a white bottle. The top-right inset visualizes the geometry-tied ROI view used on real robot; LIBERO experiments in this draft do not enable ROI.