Automated CAD Modeling Sequence Generation from Text Descriptions via Transformer-Based Large Language Models

TL;DR

This work tackles the automation of CAD modeling sequences from textual descriptions by integrating large language models with CAD-specific parametrization. It introduces a three-part approach: (i) an LLM-based semi-automated annotation pipeline for high-quality appearance and parameter descriptions, (ii) a Transformer-based dual-channel CAD generator (TCADGen) that predicts CAD command sequences with confidence scores, and (iii) CADLLM, an LLM refinement module that uses TCADGen outputs to produce a final, validated CCS. Experiments on a DeepCAD-derived dataset demonstrate substantial gains in command-sequence accuracy and reliability, with clear evidence that combining Transformer predictions and confidence-aware refinement outperforms direct generation from descriptions. The framework reduces manual effort while improving speed and precision in industrial CAD workflows, though challenges remain in annotation cost, data bias, and incorporating geometric constraints. Future work will explore geometry priors, constraint-aware learning, and scaling to broader industrial design tasks.

Abstract

Designing complex computer-aided design (CAD) models is often time-consuming due to challenges such as computational inefficiency and the difficulty of generating precise models. We propose a novel language-guided framework for industrial design automation to address these issues, integrating large language models (LLMs) with computer-automated design (CAutoD).Through this framework, CAD models are automatically generated from parameters and appearance descriptions, supporting the automation of design tasks during the detailed CAD design phase. Our approach introduces three key innovations: (1) a semi-automated data annotation pipeline that leverages LLMs and vision-language large models (VLLMs) to generate high-quality parameters and appearance descriptions; (2) a Transformer-based CAD generator (TCADGen) that predicts modeling sequences via dual-channel feature aggregation; (3) an enhanced CAD modeling generation model, called CADLLM, that is designed to refine the generated sequences by incorporating the confidence scores from TCADGen. Experimental results demonstrate that the proposed approach outperforms traditional methods in both accuracy and efficiency, providing a powerful tool for automating industrial workflows and generating complex CAD models from textual prompts. The code is available at https://jianxliao.github.io/cadllm-page/

Figures (9)

• Figure 1: The overall framework of automated CAD modeling from text descriptions, leveraging Transformer-based sequence generation and LLM-driven refinement. The innovations proposed in our work are highlighted in (a), (b), and (c).
• Figure 2: LLM-Based Semi-Automated Annotation for CAD Datasets Process. This pipeline primarily generates appearance and parameter descriptions for CAD datasets, focusing on the Description Generation and Description Quality Inspection stages.
• Figure 3: Transformer-based CAD Generator (TCADGen).
• Figure 4: Distribution of $\text{LCS}_{\text{ratio}}$ scores before and after the Reflection Optimization Process.
• Figure 5: The impact of data size on CADLLM performance. As the data size increases from 0 to 1000 samples.