Revisiting CAD Model Generation by Learning Raster Sketch

TL;DR

This paper tackles CAD model generation by replacing traditional sequence-based sketches with a raster sketch–extrusion box representation, enabling richer geometry and smoother latent-space interpolation. It introduces RECAD, a two-stage diffusion framework where one stage generates extrusion boxes and the other generates raster sketches conditioned on those boxes, with a sketch image VAE to connect latent sketches to 2D contours. The approach demonstrates strong unconditional generation performance and offers controllable generation features such as autocompletion from partial inputs, extrusion-box conditioning, and intuitive image-based editing, producing more diverse and complex CAD models than prior work. The method delivers a practical, user-friendly interface for CAD design while achieving robust generation and editing capabilities, suggesting a promising direction for integrating diffusion models into CAD pipelines.

Abstract

The integration of deep generative networks into generating Computer-Aided Design (CAD) models has garnered increasing attention over recent years. Traditional methods often rely on discrete sequences of parametric line/curve segments to represent sketches. Differently, we introduce RECAD, a novel framework that generates Raster sketches and 3D Extrusions for CAD models. Representing sketches as raster images offers several advantages over discrete sequences: 1) it breaks the limitations on the types and numbers of lines/curves, providing enhanced geometric representation capabilities; 2) it enables interpolation within a continuous latent space; and 3) it allows for more intuitive user control over the output. Technically, RECAD employs two diffusion networks: the first network generates extrusion boxes conditioned on the number and types of extrusions, while the second network produces sketch images conditioned on these extrusion boxes. By combining these two networks, RECAD effectively generates sketch-and-extrude CAD models, offering a more robust and intuitive approach to CAD model generation. Experimental results indicate that RECAD achieves strong performance in unconditional generation, while also demonstrating effectiveness in conditional generation and output editing.

Figures (11)

• Figure 1: Top: Previous approaches wu2021deepcadxu2022skexgenxu2023hierarchicalwang2024vq represent sketches using vector sequences, which often fail to produce smooth transitions in shape and topology during latent space interpolation. Bottom: RECAD learns from rasterized sketches, ensuring more plausible interpolated shapes, which results in more natural and continuous transformations between different sketches.
• Figure 2: Sample data from DeepCAD dataset. Previous methods, constrained by sequence length limitations, were restricted to learning a subset of simpler shapes within the dataset (gray). Our raster-based sketch representation enables the learning of more complex shapes (green).
• Figure 3: Framework of RECAD. A sketch image VAE, shown on the left, learns a latent feature base of rasterized input sketches. This feature base is then utilized for efficient sketch retrieval. The right panel details the two-stage diffusion process. First, extrusion boxes are progressively generated conditioned on user-specified operation and direction. Subsequently, sketch features are generated conditioned on these bounding boxes and then decoded into raster sketches using the VAE decoder. E$^{(t)}$ and S$^{(t)}$ represent visualizations of the extrusion boxes and the sketch-extrusion result at step $t$, respectively.
• Figure 4: Visual comparison on the DeepCAD dataset. Each row, from top to bottom, shows shapes generated with 1 to 5 extrusion operations, respectively. Different colors are used for clear visual distinction.
• Figure 5: Autocompleted CAD models (green) from partial inputs (gray).