Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Semantic Direct Modeling

Qiang Zou, Shuo Liu

TL;DR

Semantic Direct Modeling (SDM) bridges high-level design intent and low-level CAD edits by inserting a designer-to-semantics layer between user input and the direct-modeler. It combines speech-to-text processing, CAD-domain fine-tuned large language models, and a dynamic feature-generation module that uses cross-modal embeddings and a Transformer-based decoder to map intent to specific B-rep features and API calls for NX. The approach relies on CAD-specific prompts with chain-of-thought reasoning, textual and geometric embeddings, and multi-modal fusion to produce robust, context-aware feature labeling. Validation on mechanical design tasks, two CAD datasets, and end-to-end demonstrations show high command-interpretation accuracy and strong feature-generation performance, suggesting a practical path to more intuitive, efficient CAD workflows.

Abstract

Current direct modeling systems limit users to low-level interactions with vertices, edges, and faces, forcing designers to manage detailed geometric elements rather than focusing on high-level design intent. This paper introduces semantic direct modeling (SDM), a novel approach that lifts direct modeling from low-level geometric modifications to high-level semantic interactions. This is achieved by utilizing a large language model (LLM) fine-tuned with CAD-specific prompts, which can guide the LLM to reason through design intent and accurately interpret CAD commands, thereby allowing designers to express their intent using natural language. Additionally, SDM maps design intent to the corresponding geometric features in the CAD model through a new conditional, context-sensitive feature recognition method, which uses generative AI to dynamically assign feature labels based on design intent. Together, they enable a seamless flow from high-level design intent to low-level geometric modifications, bypassing tedious software interactions. The effectiveness of SDM has been validated through real mechanical design cases.

Semantic Direct Modeling

TL;DR

Semantic Direct Modeling (SDM) bridges high-level design intent and low-level CAD edits by inserting a designer-to-semantics layer between user input and the direct-modeler. It combines speech-to-text processing, CAD-domain fine-tuned large language models, and a dynamic feature-generation module that uses cross-modal embeddings and a Transformer-based decoder to map intent to specific B-rep features and API calls for NX. The approach relies on CAD-specific prompts with chain-of-thought reasoning, textual and geometric embeddings, and multi-modal fusion to produce robust, context-aware feature labeling. Validation on mechanical design tasks, two CAD datasets, and end-to-end demonstrations show high command-interpretation accuracy and strong feature-generation performance, suggesting a practical path to more intuitive, efficient CAD workflows.

Abstract

Current direct modeling systems limit users to low-level interactions with vertices, edges, and faces, forcing designers to manage detailed geometric elements rather than focusing on high-level design intent. This paper introduces semantic direct modeling (SDM), a novel approach that lifts direct modeling from low-level geometric modifications to high-level semantic interactions. This is achieved by utilizing a large language model (LLM) fine-tuned with CAD-specific prompts, which can guide the LLM to reason through design intent and accurately interpret CAD commands, thereby allowing designers to express their intent using natural language. Additionally, SDM maps design intent to the corresponding geometric features in the CAD model through a new conditional, context-sensitive feature recognition method, which uses generative AI to dynamically assign feature labels based on design intent. Together, they enable a seamless flow from high-level design intent to low-level geometric modifications, bypassing tedious software interactions. The effectiveness of SDM has been validated through real mechanical design cases.

Paper Structure

This paper contains 20 sections, 12 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Direct modeling
  4. Deep learning in CAD
  5. Methods
  6. The overall system design
  7. CAD domain-specific prompt engineering
  8. Dynamic feature generation
  9. Textual embedding for design semantics
  10. Geometric embedding for B-rep models
  11. Textual-geometric embedding fusion
  12. Conditional feature generation
  13. Training
  14. Results
  15. Setup
  16. ...and 5 more sections

Figures (9)

  • Figure 1: The overall architecture of SDM.
  • Figure 2: Comparison of unprompted and simple prompt responses for CAD instruction comprehension.
  • Figure 3: Chain-of-Thought guided structured parsing process for CAD design instructions.
  • Figure 4: Multi-step instruction parsing and providing the few-shots to enhance the ability of LLM in CAD domain tasks.
  • Figure 5: Illustration of B-rep model tokenization.
  • ...and 4 more figures