Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication

Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang, Ying Nian Wu, Chenfanfu Jiang

TL;DR

The introduction of Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools that ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction.

Abstract

Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional supports. To fill this gap, we introduce Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization, serving as either a refinement or a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.

Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication

TL;DR

The introduction of Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools that ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction.

Abstract

Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional supports. To fill this gap, we introduce Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization, serving as either a refinement or a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments.
Paper Structure (38 sections, 13 equations, 18 figures, 4 tables)

This paper contains 38 sections, 13 equations, 18 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Diffusion-based 3D Generation
  4. Physics-aware 3D Generation
  5. Background
  6. Score Distillation Sampling
  7. Rigid Body Dynamics
  8. Atlas3D Algorithm
  9. Physics Incorporation
  10. Standability through Differentiable Simulation
  11. Stable Equilibrium
  12. Additional Regularization
  13. Method Overview
  14. Experiments
  15. Simulation Verification
  16. ...and 23 more sections

Figures (18)

  • Figure 1: Simulation in ABD lan2022affine: (a) 3D models generated from our Atlas3D framework can stand steadily on the ground; (b) those generated from existing methods tend to fall over.
  • Figure 2: 3D-printed figurines created with Atlas3D stand stably, while those without Atlas3D have fallen down.
  • Figure 3: 2D illustration of stable equilibrium and unstable equilibrium. (a) A square is stable as a small perturbation of $\phi$ increases in $H({{\mathbf{{x}}}}_{\text{com}})$;(b) An upside-down triangle is unstable as tilting decreases $H({{\mathbf{{x}}}}_{\text{com}})$.
  • Figure 4: Comparison with Magic3D lin2023magic3d includes zoom-in views that highlight the detailed changes in geometry. Our method enhances Magic3D with physics priors to generate self-supporting meshes.
  • Figure 5: Atlas3D is also compatible with MVDream shi2023mvdream, enhancing it with stable standability.
  • ...and 13 more figures