Flash Sculptor: Modular 3D Worlds from Objects

TL;DR

Flash Sculptor introduces a divide-and-conquer pipeline for compositional 3D scene reconstruction from a single image. It decouples per-object appearance, rotation, scale, and translation, employing a coarse-to-fine rotation scheme with Nelder-Mead refinement and a robust outlier-removed translation method, complemented by background scene synthesis. The approach yields around a 3× speedup over prior compositional 3D methods while achieving state-of-the-art performance on benchmarks like T^3Bench. This framework supports scalable, parallelizable 3D scene synthesis and practical 3D editing applications.

Abstract

Existing text-to-3D and image-to-3D models often struggle with complex scenes involving multiple objects and intricate interactions. Although some recent attempts have explored such compositional scenarios, they still require an extensive process of optimizing the entire layout, which is highly cumbersome if not infeasible at all. To overcome these challenges, we propose Flash Sculptor in this paper, a simple yet effective framework for compositional 3D scene/object reconstruction from a single image. At the heart of Flash Sculptor lies a divide-and-conquer strategy, which decouples compositional scene reconstruction into a sequence of sub-tasks, including handling the appearance, rotation, scale, and translation of each individual instance. Specifically, for rotation, we introduce a coarse-to-fine scheme that brings the best of both worlds--efficiency and accuracy--while for translation, we develop an outlier-removal-based algorithm that ensures robust and precise parameters in a single step, without any iterative optimization. Extensive experiments demonstrate that Flash Sculptor achieves at least a 3 times speedup over existing compositional 3D methods, while setting new benchmarks in compositional 3D reconstruction performance. Codes are available at https://github.com/YujiaHu1109/Flash-Sculptor.

Figures (10)

• Figure 1: A scene generated by our method. Our method can generate high-quality 3D scene from a single image.
• Figure 2: Overview of our method. Given an input image, our method first separate the image into $N$ independent objects and a background. Then, all these instances undergoes disentangled four stages, i.e., appearance, rotation, scale and translation. Finally, after finishing these sub tasks, our pipeline generates a compositional 3D scene consistent with the original input.
• Figure 3: The object inpainting strategy of our method. We feed the bounding box image, mask and prompts to stable diffusion to generate inpainted objects.
• Figure 4: Illustration of the process of selecting points and removing outliers for depth alignment.
• Figure 5: Visual comparison results with other image-to-3D methods on our validation set. Other methods always lead to confusion of object structure and logical mismatch, while ours can generate high visual results. We use Hunyuan-3D as our single object reconstruction model here.