Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets

TL;DR

Seed3D 1.0 targets scalable synthesis of high-fidelity, simulation-ready 3D assets from single images to enable physics-aware embodied AI training. It combines a geometry pathway (Seed3D-VAE and Seed3D-DiT) with a textured pipeline (Seed3D-MV, Seed3D-PBR, Seed3D-UV) and is backed by a scalable data infrastructure and hardware-aware training stack. Empirical results show state-of-the-art geometry and texture generation, supported by a user study validating perceptual quality, and demonstrated applicability in robotic manipulation within Isaac Sim. The work also extends to coherent scene generation through factorized layout-to-asset assembly, moving toward practical, scalable world simulators for embodied AI. Overall, Seed3D 1.0 provides a concrete, end-to-end solution for generating physically plausible 3D content that integrates directly with physics engines and simulation pipelines.

Abstract

Developing embodied AI agents requires scalable training environments that balance content diversity with physics accuracy. World simulators provide such environments but face distinct limitations: video-based methods generate diverse content but lack real-time physics feedback for interactive learning, while physics-based engines provide accurate dynamics but face scalability limitations from costly manual asset creation. We present Seed3D 1.0, a foundation model that generates simulation-ready 3D assets from single images, addressing the scalability challenge while maintaining physics rigor. Unlike existing 3D generation models, our system produces assets with accurate geometry, well-aligned textures, and realistic physically-based materials. These assets can be directly integrated into physics engines with minimal configuration, enabling deployment in robotic manipulation and simulation training. Beyond individual objects, the system scales to complete scene generation through assembling objects into coherent environments. By enabling scalable simulation-ready content creation, Seed3D 1.0 provides a foundation for advancing physics-based world simulators. Seed3D 1.0 is now available on https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?modelId=doubao-seed3d-1-0-250928&tab=Gen3D

Figures (12)

• Figure 1: Seed3D 1.0 generates high-fidelity, simulation-ready 3D assets from single images. Individual objects generated by our system can be composed into complex scenes for simulation and robotic applications. This kitchen environment demonstrates robotic manipulation simulation with diverse generated assets. Best viewed with 8$\times$ zoom.
• Figure 2: Overview of the Seed3D 1.0 geometry generation pipeline. The framework combines a variational autoencoder named Seed3D-VAE, which is designed for compact geometry encoding and TSDF decoding, with a rectified flow–based Transformer named Seed3D-DiT to generate high-fidelity 3D shapes from input images.
• Figure 3: Seed3D-MV architecture. Left: System overview showing the multi-modal conditioning pipeline. Right: In-context multi-modal conditioning mechanism integrating geometry, reference image, and text information.
• Figure 4: The overview of Seed3D-PBR model. To handle albedo and metallic-roughness in a single DiT model, we propose a network with two-stream attention blocks. $^*$Projection contains MLPs for the computation of Q, K, and V.
• Figure 5: Data preprocessing pipeline of Seed3D 1.0. Our automated pipeline transforms raw 3D assets through format standardization, geometric deduplication, orientation canonization, and quality filtering, followed by multi-view rendering and mesh remeshing to produce training-ready datasets.