Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Model

TL;DR

This work tackles the mismatch between 2D-pretrained vision-language-action (VLA) systems and the 3D structure of the real world. It introduces Spatial Forcing (SF), which implicitly injects 3D spatial understanding by aligning intermediate visual embeddings with geometric cues from a pretrained 3D foundation model, using a joint objective $L_{SF} = L_{action} + \alpha L_{align}$. SF achieves state-of-the-art results on LIBERO and RoboTwin without relying on explicit 3D sensors, while delivering up to 3.8x faster training and improved data efficiency, validated in both simulation and real-world robotic tasks. This approach demonstrates that spatial reasoning can be transferred into 2D-trained VLAs via intermediate representation alignment, enabling robust, sensor-lean manipulation in varied environments.

Abstract

Vision-language-action (VLA) models have recently shown strong potential in enabling robots to follow language instructions and execute precise actions. However, most VLAs are built upon vision-language models pretrained solely on 2D data, which lack accurate spatial awareness and hinder their ability to operate in the 3D physical world. Existing solutions attempt to incorporate explicit 3D sensor inputs such as depth maps or point clouds, but these approaches face challenges due to sensor noise, hardware heterogeneity, and incomplete depth coverage in existing datasets. Alternative methods that estimate 3D cues from 2D images also suffer from the limited performance of depth estimators. We propose Spatial Forcing (SF), a simple yet effective alignment strategy that implicitly forces VLA models to develop spatial comprehension capabilities without relying on explicit 3D inputs or depth estimators. SF aligns intermediate visual embeddings of VLAs with geometric representations produced by pretrained 3D foundation models. By enforcing alignment at intermediate layers, SF guides VLAs to encode richer spatial representations that enhance action precision. Extensive experiments in simulation and real-world environments demonstrate that SF achieves state-of-the-art results, surpassing both 2D- and 3D-based VLAs. SF further accelerates training by up to 3.8x and improves data efficiency across diverse robotic tasks. Project page is at https://spatial-forcing.github.io/

Figures (6)

• Figure 1: Our proposed method, Spatial Forcing (SF), implicitly forces VLA models to acquire spatial-aware knowledge.(a) SF aligns intermediate visual embeddings of VLAs with geometric representations from pretrained 3D foundation models. (b) Our simple yet effective strategy yields significant improvements in training efficiency and test accuracy. (c) Depth probing proves that our SF brings spatial information into the aligned representations, further enhancing 3D perception.
• Figure 2: Comparison among different paradigms for 3D VLAs.
• Figure 3: Depth probing of the visual embeddings of VLAs. Embeddings learned solely from 2D images without alignment do not produce meaningful spatial structures. The aligned embeddings inherently contain rich spatial information, leading to better performance in depth probing.
• Figure 4: Comparisons with state-of-the-art methods on RoboTwin 2.0 benchmark.
• Figure 5: (a) We report the success rates vs. training iterations before and after representation alignment. (b) We report the success rate vs. training data before and after representation alignment. (c) The aligned representation exhibits almost the same distribution shape as the target.