Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation

Ke Niu, Haiyang Yu, Zhuofan Chen, Zhengtao Yao, Weitao Jia, Xiaodong Ge, Jingqun Tang, Benlei Cui, Bin Li, Xiangyang Xue

TL;DR

CME-CAD tackles the challenge of generating precise, editable CAD code from 2D orthographic inputs by leveraging heterogeneous expert collaboration through a two-stage training pipeline: Multi-Expert Fine-Tuning (MEFT) and Multi-Expert Reinforcement Learning (MERL). It integrates diverse pre-trained models, enforces structured reasoning via CoT, and introduces a robust reward design with executability, geometric fidelity, and coordinate-system accuracy, complemented by a hard negative sample buffering strategy. The CADExpert dataset provides 17,299 industrial-grade instances with orthographic projections, expert CoT processes, executable CADQuery code, and rendered 3D models, enabling realistic evaluation. Empirical results show notable improvements in IoU (up to 80.71%), code executability (up to 98.25%), and overall robustness, highlighting the practical potential for automating precise CAD generation in industry.

Abstract

Computer-Aided Design (CAD) is essential in industrial design, but the complexity of traditional CAD modeling and workflows presents significant challenges for automating the generation of high-precision, editable CAD models. Existing methods that reconstruct 3D models from sketches often produce non-editable and approximate models that fall short of meeting the stringent requirements for precision and editability in industrial design. Moreover, the reliance on text or image-based inputs often requires significant manual annotation, limiting their scalability and applicability in industrial settings. To overcome these challenges, we propose the Heterogeneous Collaborative Multi-Expert Reinforcement Learning (CME-CAD) paradigm, a novel training paradigm for CAD code generation. Our approach integrates the complementary strengths of these models, facilitating collaborative learning and improving the model's ability to generate accurate, constraint-compatible, and fully editable CAD models. We introduce a two-stage training process: Multi-Expert Fine-Tuning (MEFT), and Multi-Expert Reinforcement Learning (MERL). Additionally, we present CADExpert, an open-source benchmark consisting of 17,299 instances, including orthographic projections with precise dimension annotations, expert-generated Chain-of-Thought (CoT) processes, executable CADQuery code, and rendered 3D models.

CME-CAD: Heterogeneous Collaborative Multi-Expert Reinforcement Learning for CAD Code Generation

TL;DR

CME-CAD tackles the challenge of generating precise, editable CAD code from 2D orthographic inputs by leveraging heterogeneous expert collaboration through a two-stage training pipeline: Multi-Expert Fine-Tuning (MEFT) and Multi-Expert Reinforcement Learning (MERL). It integrates diverse pre-trained models, enforces structured reasoning via CoT, and introduces a robust reward design with executability, geometric fidelity, and coordinate-system accuracy, complemented by a hard negative sample buffering strategy. The CADExpert dataset provides 17,299 industrial-grade instances with orthographic projections, expert CoT processes, executable CADQuery code, and rendered 3D models, enabling realistic evaluation. Empirical results show notable improvements in IoU (up to 80.71%), code executability (up to 98.25%), and overall robustness, highlighting the practical potential for automating precise CAD generation in industry.

Abstract

Computer-Aided Design (CAD) is essential in industrial design, but the complexity of traditional CAD modeling and workflows presents significant challenges for automating the generation of high-precision, editable CAD models. Existing methods that reconstruct 3D models from sketches often produce non-editable and approximate models that fall short of meeting the stringent requirements for precision and editability in industrial design. Moreover, the reliance on text or image-based inputs often requires significant manual annotation, limiting their scalability and applicability in industrial settings. To overcome these challenges, we propose the Heterogeneous Collaborative Multi-Expert Reinforcement Learning (CME-CAD) paradigm, a novel training paradigm for CAD code generation. Our approach integrates the complementary strengths of these models, facilitating collaborative learning and improving the model's ability to generate accurate, constraint-compatible, and fully editable CAD models. We introduce a two-stage training process: Multi-Expert Fine-Tuning (MEFT), and Multi-Expert Reinforcement Learning (MERL). Additionally, we present CADExpert, an open-source benchmark consisting of 17,299 instances, including orthographic projections with precise dimension annotations, expert-generated Chain-of-Thought (CoT) processes, executable CADQuery code, and rendered 3D models.
Paper Structure (13 sections, 12 equations, 3 figures, 3 tables)

This paper contains 13 sections, 12 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CAD Code Generation
  4. Reinforcement Learning with Verifiable Rewards
  5. Method
  6. Multi-Expert Fine-Tuning (MEFT)
  7. Multi-Expert Reinforcement Learning (MERL)
  8. CADExpert
  9. Experimental Results
  10. Implementation Details
  11. Evaluation Metrics
  12. Main Results
  13. Conclusion

Figures (3)

  • Figure 1: The workflow of our method for CAD code generation.
  • Figure 2: Overall architecture of our method CME-CAD framework. $O$ represents the model output. $O^N$ denotes the concatenation of the $N$-th expert's Chain-of-Thought (CoT) and corresponding answer. $O_G^N$ represents the group of $G$ outputs generated by the $N$-th expert.
  • Figure 3: Examples from CADExpert, showcasing more complex industrial design challenges, including intricate features that align with real-world modeling difficulties.