ReCAD: Reinforcement Learning Enhanced Parametric CAD Model Generation with Vision-Language Models

TL;DR

ReCAD tackles precise parametric CAD generation from multimodal inputs by combining supervised fine-tuning of vision-language models with reinforcement learning guided by parameterized CAD code. It represents CAD as a hierarchy of primitives and trains through a two-stage process: SFT to convert hardcoded CAD scripts into parameterized code, followed by GRPO-based RL with off-policy guidance and a hierarchical primitive learning curriculum that enforces geometric and semantic fidelity. The approach achieves state-of-the-art performance on text-to-CAD and image-to-CAD tasks and demonstrates strong generalization, including zero-shot editing capabilities, suggesting practical, editable CAD generation with preserved design intent. These results indicate that exploiting PLMs’ generative priors, via parameterized code and guided RL, can significantly enhance precise, reusable CAD generation in real-world workflows.

Abstract

We present ReCAD, a reinforcement learning (RL) framework that bootstraps pretrained large models (PLMs) to generate precise parametric computer-aided design (CAD) models from multimodal inputs by leveraging their inherent generative capabilities. With just access to simple functional interfaces (e.g., point coordinates), our approach enables the emergence of complex CAD operations (e.g., pattern replication and mirror). This stands in contrast to previous methods, which typically rely on knowledge injected through supervised fine-tuning (SFT), offer limited support for editability, and fail to exploit the strong generative priors of PLMs. Specifically, the ReCAD framework begins by fine-tuning vision-language models (VLMs) to equip them with basic CAD model generation capabilities, where we rewrite CAD scripts into parameterized code that is leveraged to generate accurate textual descriptions for supervision. Then, we propose a novel RL strategy that incorporates parameterized code as guidance to enhance the model's reasoning on challenging questions. Furthermore, we employ a hierarchical primitive learning process to progressively teach structured and compositional skills under a unified reward function that ensures both geometric accuracy and semantic fidelity. ReCAD sets a new state-of-the-art in both text-to-CAD and image-to-CAD tasks, significantly improving geometric accuracy across in-distribution and out-of-distribution settings. In the image-to-CAD task, for instance, it reduces the mean Chamfer Distance from 73.47 to 29.61 (in-distribution) and from 272.06 to 80.23 (out-of-distribution), outperforming existing baselines by a substantial margin.

Figures (9)

• Figure 1: Visualization of the 3D model generated by our method (ReCAD-VL) based on a single image or textual description (abstract or precise). Our method enables the generation of complex and diverse parametric CAD models in code representation, which allows precise parameter control and facilitates easy editing and reuse.
• Figure 2: Overview of the proposed ReCAD framework. (1) A CAD model is represented as a primitive hierarchy, where each primitive is converted into parameterized code, which is further leveraged to generate precise textual descriptions. (2) We first perform supervised fine-tuning (SFT) for basic CAD code generation, then optimize the model via reinforcement learning guided by parameterized code, with reward functions enforcing both geometric fidelity and feature-level consistency.
• Figure 3: Dialog results on CAD generation and zero-shot CAD-related tasks. Despite being trained exclusively on CAD generation tasks, ReCAD-VL exhibits impressive zero-shot performance across multiple CAD-related tasks, including understanding, editing, debugging, and descriptive captioning. More detailed qualitative results are provided in the Appendix.
• Figure 4: Qualitative comparison of different methods on text-to-CAD and image-to-CAD tasks.
• Figure A1: Visual results of ReCAD-VL on the text-to-CAD task.