TransCAD: A Hierarchical Transformer for CAD Sequence Inference from Point Clouds

TL;DR

TransCAD introduces a hierarchical, end-to-end transformer framework that maps point clouds to explicit CAD construction sequences. It coreises a two-stage decoding: a high-level loop-extrusion embedding followed by dedicated loop and extrusion decoders, with a loop refiner to recover unquantized parameters. The paper also proposes a principled APCS metric and CSSS framework to evaluate CAD sequences beyond geometry similarity. Across DeepCAD and Fusion360 datasets, TransCAD achieves state-of-the-art sequence recovery and demonstrates robustness to realistic point-cloud perturbations, bridging the gap to practical CAD reverse engineering.

Abstract

3D reverse engineering, in which a CAD model is inferred given a 3D scan of a physical object, is a research direction that offers many promising practical applications. This paper proposes TransCAD, an end-to-end transformer-based architecture that predicts the CAD sequence from a point cloud. TransCAD leverages the structure of CAD sequences by using a hierarchical learning strategy. A loop refiner is also introduced to regress sketch primitive parameters. Rigorous experimentation on the DeepCAD and Fusion360 datasets show that TransCAD achieves state-of-the-art results. The result analysis is supported with a proposed metric for CAD sequence, the mean Average Precision of CAD Sequence, that addresses the limitations of existing metrics.

Figures (16)

• Figure 1: The sequential process of CAD modeling. A CAD sequence $\mathbf C$ can be decomposed into a hierarchical structure. The highest conceptual level is a sequence of sketch $\mathbf s$ and extrusion $\mathbf e$. A sketch can be made of one or more loops $\mathbf \rho$. Each loop can be further decomposed into loop primitives, circle, arc and line. Each loop primitive can be described by a fixed number of parameters as shown on the right panel.
• Figure 1: Example of a CAD model and its CAD sequence in the formulation proposed. The top panels depicts the CAD construction process. The middle panel shows the corresponding high-level loop-extrusion sequence that is predicted as part of the hierarchical learning. The low-level parameters of each loop primitive and extrusion are displayed in the bottom panel.
• Figure 2: TransCAD model architecture. TransCAD is a hierarchical network composed of the following components: a point cloud encoder, a loop-extrusion decoder that predicts a high-level sequence which is then decoded by a loop decoder and an extrusion decoder. The predicted quantized loop parameters are then corrected by a loop refiner.
• Figure 2: Examples of duplicate CAD models from the DeepCAD wu2021deepcad and Fusion360 willis2021fusion datasets. On the top left panel, CAD models from the DeepCAD train set with geometrical duplicates in the test set are shown. Similarly, the right panel presents geometrical duplicates present in the Fusion360 willis2021fusion dataset. CAD models with identical CAD sequences, i.e. sequence duplicates, are displayed in the bottom left panel.
• Figure 3: Two examples outlining the limitations of existing evaluation metrics. Left panel: the ground truth sequence (one sketch-extrusion) is a correct subset of the predicted sequence (three sketch-extrusions). The DeepCAD wu2021deepcad metrics result in an accuracy of $1$ for both commands and parameters. On the other hand, our proposed metric takes into account the over predicted sequence elements and the APCS is 0.031. The right panel showcases the limitations of the $CD$ as a similarity measure. While the ground truth and predicted shapes are both composed of three extruded circle sketches, they are different in shape. However, the $CD$ between the two shapes falls within the uncertainty range of $\pm 0.3$. Note that the uncertainty in the $CD$ measurement is estimated by taking the average $CD$ between all the test samples and themselves.