Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RoboBrain 2.5: Depth in Sight, Time in Mind

Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, Mingyu Cao, Sixiang Chen, Zhe Li, Mengzhen Liu, Zixiao Wang, Shanyu Rong, Yaoxu Lyu, Zhongxia Zhao, Peterson Co, Yibo Li, Yi Han, Shaoxuan Xie, Guocai Yao, Songjing Wang, Leiduo Zhang, Xi Yang, Yance Jiao, Donghai Shi, Kunchang Xie, Shaokai Nie, Chunlei Men, Yonghua Lin, Zhongyuan Wang, Tiejun Huang, Shanghang Zhang

TL;DR

RoboBrain 2.5 tackles the gap between high-level semantic reasoning and physically grounded manipulation by introducing depth-aware 3D spatial reasoning and dense step-aware temporal modeling. It advances from 2D grounding to depth-enabled coordinates $(u,v,d)$ and formalizes complete 3D manipulation traces $ au=igl\{p_t\bigr\}$ under absolute metric constraints, while also providing dense temporal progress signals through hop-based labeling and multi-view fusion. The work follows a two-stage training strategy over a mixed 12.4M dataset and implements cross- accelerator infrastructure to enable scalable multimodal learning, achieving state-of-the-art results on 2D/3D spatial benchmarks and robust temporal estimation across real, simulated, and human data. These capabilities translate to more reliable, physically grounded embodied agents with broad deployment potential and set the stage for unified world models and self-improving data engines in robotics.

Abstract

We introduce RoboBrain 2.5, a next-generation embodied AI foundation model that advances general perception, spatial reasoning, and temporal modeling through extensive training on high-quality spatiotemporal supervision. Building upon its predecessor, RoboBrain 2.5 introduces two major capability upgrades. Specifically, it unlocks Precise 3D Spatial Reasoning by shifting from 2D pixel-relative grounding to depth-aware coordinate prediction and absolute metric constraint comprehension, generating complete 3D manipulation traces as ordered keypoint sequences under physical constraints. Complementing this spatial precision, the model establishes Dense Temporal Value Estimation that provides dense, step-aware progress prediction and execution state understanding across varying viewpoints, producing stable feedback signals for downstream learning. Together, these upgrades extend the framework toward more physically grounded and execution-aware embodied intelligence for complex, fine-grained manipulation. The code and checkpoints are available at project website: https://superrobobrain.github.io

RoboBrain 2.5: Depth in Sight, Time in Mind

TL;DR

RoboBrain 2.5 tackles the gap between high-level semantic reasoning and physically grounded manipulation by introducing depth-aware 3D spatial reasoning and dense step-aware temporal modeling. It advances from 2D grounding to depth-enabled coordinates and formalizes complete 3D manipulation traces under absolute metric constraints, while also providing dense temporal progress signals through hop-based labeling and multi-view fusion. The work follows a two-stage training strategy over a mixed 12.4M dataset and implements cross- accelerator infrastructure to enable scalable multimodal learning, achieving state-of-the-art results on 2D/3D spatial benchmarks and robust temporal estimation across real, simulated, and human data. These capabilities translate to more reliable, physically grounded embodied agents with broad deployment potential and set the stage for unified world models and self-improving data engines in robotics.

Abstract

We introduce RoboBrain 2.5, a next-generation embodied AI foundation model that advances general perception, spatial reasoning, and temporal modeling through extensive training on high-quality spatiotemporal supervision. Building upon its predecessor, RoboBrain 2.5 introduces two major capability upgrades. Specifically, it unlocks Precise 3D Spatial Reasoning by shifting from 2D pixel-relative grounding to depth-aware coordinate prediction and absolute metric constraint comprehension, generating complete 3D manipulation traces as ordered keypoint sequences under physical constraints. Complementing this spatial precision, the model establishes Dense Temporal Value Estimation that provides dense, step-aware progress prediction and execution state understanding across varying viewpoints, producing stable feedback signals for downstream learning. Together, these upgrades extend the framework toward more physically grounded and execution-aware embodied intelligence for complex, fine-grained manipulation. The code and checkpoints are available at project website: https://superrobobrain.github.io
Paper Structure (30 sections, 12 equations, 13 figures, 4 tables)

This paper contains 30 sections, 12 equations, 13 figures, 4 tables.

Table of Contents

  1. Introduction
  2. New Feature
  3. Precise 3D Spatial Reasoning
  4. 3D Spatial Referring, Measuring, and Tracing
  5. 3D Task Formulation
  6. Dense Temporal Value Estimation
  7. Hop-wise Progress Construction
  8. Multi-Perspective Progress Fusion
  9. Bi-directional Consistency Checking
  10. Training Data
  11. General MLLM Data
  12. Spatial Reasoning Data
  13. Temporal Prediction Data
  14. Training Strategy
  15. Stage 1: Foundational Spatiotemporal Learning
  16. ...and 15 more sections

Figures (13)

  • Figure 1: New Features of RoboBrain 2.5. Top: Precise 3D spatial reasoning with depth-aware grounding, metric measuring, and full manipulation trace generation under physical constraints. Bottom: Dense temporal value estimation for step-aware progress/regress prediction from state transitions across viewpoints and tasks; radar plots summarize performance gains on 2D/3D spatial and temporal benchmarks.
  • Figure 2: Training Data Distribution for RoboBrain 2.5. The left pie chart illustrates the hierarchical composition of the dataset, structured into Temporal (red), General (teal), and Spatial (blue) domains. The right bar chart displays the sample count for each specific sub-task on a logarithmic scale, highlighting the extensive scale of Dense Value Estimation, High-Quality General Data, and 3D Spatial Reasoning.
  • Figure 3: Visualization of TraceSpatial-Bench Rollouts and RoboBrain 2.5’s Predicted Traces. The red mask marks the ground-truth starting point, the purple 3D bounding box represents the ground-truth endpoint, and the 2D projection of RoboBrain 2.5’s predicted 3D spatial trace is displayed.
  • Figure 4: Visualization of TraceSpatial-Bench Rollouts and RoboBrain 2.5’s Predicted Traces. The red mask marks the ground-truth starting point, the purple 3D bounding box represents the ground-truth endpoint, and the 2D projection of RoboBrain 2.5’s predicted 3D spatial trace is displayed.
  • Figure 5: Visualization of TraceSpatial-Bench Rollouts and RoboBrain 2.5’s Predicted Traces. The red mask marks the ground-truth starting point, the purple 3D bounding box represents the ground-truth endpoint, and the 2D projection of RoboBrain 2.5’s predicted 3D spatial trace is displayed.
  • ...and 8 more figures