GACO-CAD: Geometry-Augmented and Conciseness-Optimized CAD Model Generation from Single Image

TL;DR

This work tackles single-view editable CAD generation by introducing GACO-CAD, a two-stage post-training framework. It first injects dense geometric priors—depth and surface normals—into supervised fine-tuning to enhance 3D spatial reasoning, then applies reinforcement learning with a group-length reward to simultaneously improve geometric fidelity and modeling conciseness. The approach achieves state-of-the-art results on DeepCAD and Fusion360 in terms of code validity, geometric accuracy, and brevity of the generated CAD sequences. The proposed methodology has practical impact for industrial design, reducing manual modeling effort while increasing editability of generated CAD models.

Abstract

Generating editable, parametric CAD models from a single image holds great potential to lower the barriers of industrial concept design. However, current multi-modal large language models (MLLMs) still struggle with accurately inferring 3D geometry from 2D images due to limited spatial reasoning capabilities. We address this limitation by introducing GACO-CAD, a novel two-stage post-training framework. It is designed to achieve a joint objective: simultaneously improving the geometric accuracy of the generated CAD models and encouraging the use of more concise modeling procedures. First, during supervised fine-tuning, we leverage depth and surface normal maps as dense geometric priors, combining them with the RGB image to form a multi-channel input. In the context of single-view reconstruction, these priors provide complementary spatial cues that help the MLLM more reliably recover 3D geometry from 2D observations. Second, during reinforcement learning, we introduce a group length reward that, while preserving high geometric fidelity, promotes the generation of more compact and less redundant parametric modeling sequences. A simple dynamic weighting strategy is adopted to stabilize training. Experiments on the DeepCAD and Fusion360 datasets show that GACO-CAD achieves state-of-the-art performance under the same MLLM backbone, consistently outperforming existing methods in terms of code validity, geometric accuracy, and modeling conciseness.

Figures (6)

• Figure 1: Demenstration of various CAD models generated by GACO-CAD.
• Figure 2: Main Pipeline of Two-stage Training. Starting from pre-trained weights, we obtain the final model through two training stages: SFT and RL. In the SFT stage, we perform full fine-tuning with depth and normal priors; in the RL stage, we optimize with the GSPO algorithm incorporating a length-reward rule. After training, the model takes a single-view image and simple text as input and generates target Python code, whose execution yields the final B-rep file.
• Figure 3: The Role of Deep Prior in Monocular 3D Reconstruction. When the camera parameters and the limited output model bounding box are defined, the introduction of the relative depth map theoretically provides sufficient conditions for accurately reconstructing the 3D model of the visible part in a single view.
• Figure 4: SFT Pipeline. We use Moge-2 and DepthAnything v2 to estimate normal map and depth map. All three images are processed by the shared ViT encoder, concatenated with the text, and fed into the LLM and get the first-stage model: GACO-SFT.
• Figure 5: Reinforcement Learning with Group Length Reward Rule. We conducted RL training in GACO-SFT. In the generated responses, those with high IoU accuracy would be rewarded based on the group relative token length, while responses with low IoU and below the length threshold would be penalized. Finally, we obtained the two-stage model: GACO-RL.