Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model

Christopher Nguyen, William Nguyen, Atsushi Suzuki, Daisuke Oku, Hong An Phan, Sang Dinh, Zooey Nguyen, Anh Ha, Shruti Raghavan, Huy Vo, Thang Nguyen, Lan Nguyen, Yoshikuni Hirayama

TL;DR

Through fine-tuning a pre-trained LLM using the authors' curated dataset, it is shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks, paving the way for further research and applications in the semiconductor domain.

Abstract

Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong

SemiKong: Curating, Training, and Evaluating A Semiconductor Industry-Specific Large Language Model

TL;DR

Through fine-tuning a pre-trained LLM using the authors' curated dataset, it is shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks, paving the way for further research and applications in the semiconductor domain.

Abstract

Large Language Models (LLMs) have demonstrated the potential to address some issues within the semiconductor industry. However, they are often general-purpose models that lack the specialized knowledge needed to tackle the unique challenges of this sector, such as the intricate physics and chemistry of semiconductor devices and processes. SemiKong, the first industry-specific LLM for the semiconductor domain, provides a foundation that can be used to develop tailored proprietary models. With SemiKong 1.0, we aim to develop a foundational model capable of understanding etching problems at an expert level. Our key contributions include (a) curating a comprehensive corpus of semiconductor-related texts, (b) creating a foundational model with in-depth semiconductor knowledge, and (c) introducing a framework for integrating expert knowledge, thereby advancing the evaluation process of domain-specific AI models. Through fine-tuning a pre-trained LLM using our curated dataset, we have shown that SemiKong outperforms larger, general-purpose LLMs in various semiconductor manufacturing and design tasks. Our extensive experiments underscore the importance of developing domain-specific LLMs as a foundation for company- or tool-specific proprietary models, paving the way for further research and applications in the semiconductor domain. Code and dataset will be available at https://github.com/aitomatic/semikong

Paper Structure

This paper contains 18 sections, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Semiconductor Manufacturing and Design
  3. Use of LLMs in Semiconductors
  4. Purpose and Scope
  5. Related Works
  6. Artificial Intelligence (AI) in semiconductor manufacturing
  7. LLMs in the semiconductor industry
  8. LLM as an evaluator
  9. Semiconductor Ontology
  10. SemiKong: Semiconductor Industry Specific LLM
  11. Data Curation
  12. Model tranining
  13. The proposed method for evaluating LLM in semiconductor manufacturing
  14. Experimental Result
  15. Implementation details
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Comparison of SemiKong and commercial models. SemiKong is a open source foundation model but achieved comparable performance on E&B (Efficiency and Brevity), C&D (Clarity and Directness) with other commercial models and significantly outperformed these products in PIU (Practicality and Immediate Usability), LFC (Logical Flow and Coherence), EEC (Expert-to-Expert Communication), UES (Use of Examples and Specificity).
  • Figure 2: The evaluation benchmark development pipeline.