Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Utonia: Toward One Encoder for All Point Clouds

Yujia Zhang, Xiaoyang Wu, Yunhan Yang, Xianzhe Fan, Han Li, Yuechen Zhang, Zehao Huang, Naiyan Wang, Hengshuang Zhao

TL;DR

Utonia is presented, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos.

Abstract

We dream of a future where point clouds from all domains can come together to shape a single model that benefits them all. Toward this goal, we present Utonia, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos. Despite their distinct sensing geometries, densities, and priors, Utonia learns a consistent representation space that transfers across domains. This unification improves perception capability while revealing intriguing emergent behaviors that arise only when domains are trained jointly. Beyond perception, we observe that Utonia representations can also benefit embodied and multimodal reasoning: conditioning vision-language-action policies on Utonia features improves robotic manipulation, and integrating them into vision-language models yields gains on spatial reasoning. We hope Utonia can serve as a step toward foundation models for sparse 3D data, and support downstream applications in AR/VR, robotics, and autonomous driving.

Utonia: Toward One Encoder for All Point Clouds

TL;DR

Utonia is presented, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos.

Abstract

We dream of a future where point clouds from all domains can come together to shape a single model that benefits them all. Toward this goal, we present Utonia, a first step toward training a single self-supervised point transformer encoder across diverse domains, spanning remote sensing, outdoor LiDAR, indoor RGB-D sequences, object-centric CAD models, and point clouds lifted from RGB-only videos. Despite their distinct sensing geometries, densities, and priors, Utonia learns a consistent representation space that transfers across domains. This unification improves perception capability while revealing intriguing emergent behaviors that arise only when domains are trained jointly. Beyond perception, we observe that Utonia representations can also benefit embodied and multimodal reasoning: conditioning vision-language-action policies on Utonia features improves robotic manipulation, and integrating them into vision-language models yields gains on spatial reasoning. We hope Utonia can serve as a step toward foundation models for sparse 3D data, and support downstream applications in AR/VR, robotics, and autonomous driving.
Paper Structure (15 sections, 5 equations, 4 figures, 18 tables)

This paper contains 15 sections, 5 equations, 4 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Pilot Study: What Prevents Unification?
  3. Utonia
  4. Causal Modality Blinding
  5. Perceptual Granularity Rescale
  6. RoPE Bridges Granularity-Aligned Coordinates
  7. Experiments
  8. Ablations
  9. Downstream Applications
  10. Related Works
  11. Conclusion and Future
  12. Additional Implementation
  13. Data Preparation and Training Details
  14. RoPE Implementation Details
  15. Downstream Tasks Implementation

Figures (4)

  • Figure 1: Cross-domain semantic similarity. Human perception operates at a fixed angular resolution, resulting in similar perception granularity between a close small toy car and a far-away real car, which motivates semantic matching at a canonical granularity across domains. Utonia representations exhibit high similarity between the toy car from object CAD and real cars in outdoor scenes, while the previous SOTA Concerto fails to align.
  • Figure 2: Gravity priors influence. Scene-level data have a strong z-axis up prior. Utonia steps further to erase such assumptions by including rotation-invariant objects with strong SE(3) augmentations into pretraining datasets.
  • Figure 3: Overview of Utonia. Utonia introduces three critical improvements to the point cloud SSL pipeline. Cross-domain data: jointly training on object-centric, indoor, and outdoor point clouds. RoPE-Enhanced Point Transformer V3: Strengthening spatial encoding and cross-domain transferability via RoPE on granularity-aligned coordinates and domain-prior erasure. Broader evaluation: extending beyond standard perception tasks to spatial reasoning, robotic manipulation, and open-world part segmentation.
  • Figure 4: Utonia features in cluttered manipulation scenes. Utonia can separate objects from supporting surfaces and remain coherent under occlusion and partial observations, providing geometry-aware cues that are useful for downstream grasping and motion planning.