A Generalizable Framework for Building Executable Domain-Specific LLMs under Data Scarcity: Demonstration on Semiconductor TCAD Simulation

TL;DR

This work tackles data scarcity in specialized domains where outputs must be executable scripts by introducing a schema-first alignment framework. It combines large-scale QA synthesis, an IR->DPO code-alignment pipeline, and optional RAG to train compact domain-specific LLMs, demonstrated on TCAD as TcadGPT and extended to the Elmer FEM solver. The approach yields a 1.5M QA corpus, a 264-question TCAD benchmark, and an 80% syntax pass rate on a code-executability test, significantly outperforming general-purpose baselines. The results reveal that executable LLMs can be built reproducibly in data-scarce domains, with RAG providing mixed benefits depending on domain specialization, and suggest a reusable pattern for tool-executable LLMs across scientific and engineering domains.

Abstract

Scientific and engineering verticals often suffer from data scarcity and strict executability requirements: models must generate not only fluent text, but also syntactically valid, tool-compilable scripts. We present a schema-first alignment framework for building compact, executable domain-specific LLMs in low-resource settings. The framework integrates three core components: (i) large-scale synthetic QA data generation from expert documentation to instill foundational domain knowledge; (ii) a code-centric IR->DPO workflow that converts verified tool decks into interpretable intermediate representations (IR), performs equivalence-preserving diversification, and constructs preference pairs to directly optimize instruction compliance and code executability; and (iii) a controlled evaluation of Retrieval-Augmented Generation (RAG), showing that while RAG benefits general LLMs, it can marginally degrade the performance of already domain-aligned models. We demonstrate the framework by instantiating TcadGPT for semiconductor Technology Computer-Aided Design (TCAD). Using 1.5M synthetic QA pairs and an IR-driven DPO dataset, TcadGPT attains 85.6% semantic accuracy and an 80.0% syntax pass rate on SDE executability tests, substantially outperforming state-of-the-art general LLMs such as GPT-4o. To probe portability beyond TCAD, we apply the same recipe to the open-source FEM solver Elmer, observing consistent improvements in script-level success rates over general-purpose baselines. All datasets, benchmarks, and code (including P1, P2, and IR->DPO) are released for reproducibility. Together, these results suggest that the proposed framework provides a robust and reproducible path toward executable LLMs in specialized, data-scarce professional domains.

Figures (10)

• Figure 1: Executable result produced by a model-generated SDE deck. A planar MOSFET structure constructed from a natural-language instruction and rendered by our model into an SDE script, then executed to completion without manual edits. Shown is the dopant distribution (DopingConcentration, cm$^{-3}$) after mesh generation and export (.tdr/.bnd). The example includes multiple materials (Silicon, SiO$_2$, PolySi, Si$_3$N$_4$), contact definitions (gate/source/drain/substrate), region-specific doping profiles, and multi-level mesh refinement. This result demonstrates both instruction compliance and tool executability, motivating the schema-first IR$\rightarrow$DPO alignment strategy introduced later in this paper.
• Figure 2: Representative examples from the TCAD benchmark, covering multiple question types including operational, conceptual, and model-related. These illustrate the diversity and depth of domain understanding required for high-performance models.
• Figure 3: Distribution of question types in the TCAD benchmark. The six major categories—Physical Model, Simulation, SDE, SProcess, SDevice, and SVisual—cover all key simulation and modeling tasks encountered in real-world workflows.
• Figure 4: Composition of the synthetic training dataset by data source (User Guides, Training Docs, Textbooks) and data generation pipeline (Pipeline 1, Pipeline 2). The outer ring shows the total contribution of each source, while the inner ring distinguishes the two QA generation pipelines.
• Figure 5: Examples of JSON QA Pairs used for fine-tuning.