CADDreamer: CAD Object Generation from Single-view Images

TL;DR

CADDreamer tackles the challenge of converting single-view images into manufacturable CAD B-reps by pairing a primitive-aware, cross-domain diffusion process with a geometry- and topology-preserving extraction pipeline. The method first generates a complete mesh and segments it into primitive patches using a diffusion model conditioned on normal and semantic primitive maps, then refines primitive parameters and enforces important geometric relationships through a stitching-based optimization and topology-preserving intersections to yield a watertight B-rep. It demonstrates superior mesh fidelity, accurate primitive segmentation, and robust CAD topology on both synthetic and real images, outperforming state-of-the-art single-view reconstruction baselines. This approach enables compact, structured CAD reconstructions from imagery, with practical impact for design, manufacturing, and rapid CAD prototyping from photographs or simple sketches.

Abstract

Diffusion-based 3D generation has made remarkable progress in recent years. However, existing 3D generative models often produce overly dense and unstructured meshes, which stand in stark contrast to the compact, structured, and sharply-edged Computer-Aided Design (CAD) models crafted by human designers. To address this gap, we introduce CADDreamer, a novel approach for generating boundary representations (B-rep) of CAD objects from a single image. CADDreamer employs a primitive-aware multi-view diffusion model that captures both local geometric details and high-level structural semantics during the generation process. By encoding primitive semantics into the color domain, the method leverages the strong priors of pre-trained diffusion models to align with well-defined primitives. This enables the inference of multi-view normal maps and semantic maps from a single image, facilitating the reconstruction of a mesh with primitive labels. Furthermore, we introduce geometric optimization techniques and topology-preserving extraction methods to mitigate noise and distortion in the generated primitives. These enhancements result in a complete and seamless B-rep of the CAD model. Experimental results demonstrate that our method effectively recovers high-quality CAD objects from single-view images. Compared to existing 3D generation techniques, the B-rep models produced by CADDreamer are compact in representation, clear in structure, sharp in edges, and watertight in topology.

Figures (14)

• Figure 1: A gallery of our reconstructed CAD models (middle and right) from single-view RGB images (left). The reconstructed shapes are shown in light blue (middle), while their topological representations of B-rep vertices and edges are shown in green (right).
• Figure 2: The pipeline of CADDreamer. In the first module, the given single-view RGB image is converted as a normal map. Using the normal map as input, the generation module uses a diffusion process to generate multi-view normal and semantic primitive maps. Inputting the multi-view normal map into Neus wang2021neus, we obtain 3D meshes; back-projecting semantic primitive maps into 3D meshes, we segmented the mesh into several patches with a Graph Cut process. In the second module, geometric optimization corrects the noisy primitive parameters, while the topology-preserving extraction computes their topology-guided intersections and reconstructs a watertight B-rep CAD model.
• Figure 3: An example of the Graph Cut process to obtain complete 3D mesh patches representing primitives. (a) depicts the mesh patches generated by feature-line cut and back-projection. (b) illustrates the process of Graph Cut algorithm to obtain mesh patches corresponding to primitives.
• Figure 4: Four key primitive relationships: (a-b) cylinder-plane intersection and perpendicularity; (c) parallel cylinders; (d) collinear cylinder and torus. Incorrect relationships (columns 2 & 4) yield flawed intersections, while correct ones (columns 3 & 5) produce accurate results.
• Figure 5: An example of primitive stitching process with four stitching vertices. (a) shows the patch boundaries on reconstructed mesh, (b) represents the sampled stitching vertices (k=4), (c) illustrates the initial primitives and the projected points of stitching vertices, (d) depicts the stitching result after the first step of optimization, (e) shows the result at the 100th step, and (f) represents the final stitching result.