CAD-Tokenizer: Towards Text-based CAD Prototyping via Modality-Specific Tokenization

TL;DR

CAD-Tokenizer is proposed, a framework that represents CAD data with modality-specific tokens using a sequence-based VQ-VAE with primitive-level pooling and constrained decoding, which significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purpose LLMs and task-specific baselines.

Abstract

Computer-Aided Design (CAD) is a foundational component of industrial prototyping, where models are defined not by raw coordinates but by construction sequences such as sketches and extrusions. This sequential structure enables both efficient prototype initialization and subsequent editing. Text-guided CAD prototyping, which unifies Text-to-CAD generation and CAD editing, has the potential to streamline the entire design pipeline. However, prior work has not explored this setting, largely because standard large language model (LLM) tokenizers decompose CAD sequences into natural-language word pieces, failing to capture primitive-level CAD semantics and hindering attention modules from modeling geometric structure. We conjecture that a multimodal tokenization strategy, aligned with CAD's primitive and structural nature, can provide more effective representations. To this end, we propose CAD-Tokenizer, a framework that represents CAD data with modality-specific tokens using a sequence-based VQ-VAE with primitive-level pooling and constrained decoding. This design produces compact, primitive-aware representations that align with CAD's structural nature. Applied to unified text-guided CAD prototyping, CAD-Tokenizer significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purpose LLMs and task-specific baselines.

Figures (12)

• Figure 1: Overview of the unified text-based CAD prototyping task, which includes the Text-to-CAD generation and text-based CAD Editing. Our approach introduces a CAD-specific tokenization method that produces primitive-level tokens instead word pieces. This paradigm allows LLMs to capture relationships among CAD primitives more effectively, improves compression and performance.
• Figure 2: Overview of the CAD-Tokenizer framework. (1) A primitive-based VQ-VAE tokenizes CAD sequences into discrete primitive tokens. (2) Adapter modules align these tokens with the LLM embedding space. (3) The backbone LLM is fine-tuned using CAD-specific tokens. (4) At inference, an FSA-guided sampling strategy ensures syntactically valid CAD generation.
• Figure 2: The ablation study on the reconstruction quality of the CAD-Tokenizer variants and HNC-CAD. Only the sketch scores are reported for the F1 score because the CAD-Tokenizer encodes objects by sketch-extrusion pairs and always achieves full score for this sub-metric. CAD-Tokenizer(curve) is the default variant that we reported in main results.
• Figure 3: An overview of our FSA design. At each step, the FSA receives an input action, transitions to the corresponding new state (i.e., updates its internal state), and returns the mask(s) associated with that new state (node). The FSA map is specific to our CAD sequence representation, which can be found in Appendix \ref{['sec:apdx:representation']}.
• Figure 4: The Compression ratio of the different tokenization algorithms. The compression ratio is 100% for no compression.