AnalogSeeker: An Open-source Foundation Language Model for Analog Circuit Design

TL;DR

AnalogSeeker introduces an open-source foundation language model tailored for analog circuit design by coupling a domain-knowledge corpus with granular distillation into QTSA-formatted data and a neighborhood self-constrained fine-tuning (NSC-SFT) regime. The approach mitigates data scarcity by curating textbooks, decomposing content into learning nodes, and extracting explicit reasoning via a multi-agent distillation framework, yielding a 112.65M-token labeled dataset from 7.26M unlabeled tokens. NSC-SFT constrains updates with a KL divergence term to preserve the reference model's semantic space while injecting domain knowledge, achieving 85.04% accuracy on AMSBench-TQA and outperforming many baselines. The work demonstrates viable domain adaptation for analog circuit design, offers practical training insights, and provides an open-source model to spur further research in AI-assisted EDA tools.

Abstract

In this paper, we propose AnalogSeeker, an effort toward an open-source foundation language model for analog circuit design, with the aim of integrating domain knowledge and giving design assistance. To overcome the scarcity of data in this field, we employ a corpus collection strategy based on the domain knowledge framework of analog circuits. High-quality, accessible textbooks across relevant subfields are systematically curated and cleaned into a textual domain corpus. To address the complexity of knowledge of analog circuits, we introduce a granular domain knowledge distillation method. Raw, unlabeled domain corpus is decomposed into typical, granular learning nodes, where a multi-agent framework distills implicit knowledge embedded in unstructured text into question-answer data pairs with detailed reasoning processes, yielding a fine-grained, learnable dataset for fine-tuning. To address the unexplored challenges in training analog circuit foundation models, we explore and share our training methods through both theoretical analysis and experimental validation. We finally establish a fine-tuning-centric training paradigm, customizing and implementing a neighborhood self-constrained supervised fine-tuning algorithm. This approach enhances training outcomes by constraining the perturbation magnitude between the model's output distributions before and after training. In practice, we train the Qwen2.5-32B-Instruct model to obtain AnalogSeeker, which achieves 85.04% accuracy on AMSBench-TQA, the analog circuit knowledge evaluation benchmark, with a 15.67% point improvement over the original model and is competitive with mainstream commercial models. Furthermore, AnalogSeeker also shows effectiveness in the downstream operational amplifier design task. AnalogSeeker is open-sourced at https://huggingface.co/analogllm/analogseeker for research use.

Figures (8)

• Figure 1: The overall workflow and contribution of this paper.
• Figure 2: An example of the granular decomposition of the cleaned corpus. From left to right, the granularity becomes progressively finer.
• Figure 3: The multi-agent framework for knowledge distillation. The upper half is the data generation stage, while the lower half is the post-processing stage.
• Figure 4: An easy example of QTSA data pairs. Some content details have been abbreviated using ellipses due to space limitations.
• Figure 5: The implementation of the proposed NSC-SFT algorithm. This diagram specifically demonstrates the data flow during the forward propagation and gradient backpropagation process.