The Graph's Apprentice: Teaching an LLM Low Level Knowledge for Circuit Quality Estimation

TL;DR

This work addresses the challenge of rapidly predicting post-synthesis circuit quality (area and delay) directly from HDL code to accelerate RTL-level design iterations. It introduces VeriDistill, a decoder trained on frozen Verilog-focused LLM representations and regularized by a LUT-GNN teacher trained on LUT graphs, with a loss that blends supervised QoR prediction and cross-modal knowledge distillation. The approach outperforms state-of-the-art RTL-level baselines, including AST-based models, and demonstrates stronger generalization to out-of-distribution circuits such as those in the OpenABCD benchmark. By leveraging high-level language representations together with low-level circuit insights, VeriDistill provides instant HDL feedback and highlights the value of cross-representation alignment in hardware-aware learning.

Abstract

Logic synthesis is a crucial phase in the circuit design process, responsible for transforming hardware description language (HDL) designs into optimized netlists. However, traditional logic synthesis methods are computationally intensive, restricting their iterative use in refining chip designs. Recent advancements in large language models (LLMs), particularly those fine-tuned on programming languages, present a promising alternative. This work proposes augmenting LLMs with predictor networks trained to estimate circuit quality directly from HDL code. To enhance performance, the model is regularized using embeddings from graph neural networks (GNNs) trained on Look-Up Table (LUT) graphs, thereby incorporating lower-level circuit insights. The proposed method demonstrates superior performance compared to existing graph-based RTL-level estimation techniques on the established benchmark OpenABCD, while providing instant feedback on HDL code quality.

Figures (5)

• Figure 1: Overview of the training and inference pipeline. During training, LUT graphs, and the area/delay labels are used to train the model. During inference, only the source Verilog is required to generate the post-synthesis area/delay prediction.
• Figure 2: The training procedure. The Verilog training examples are passed to the VeriDistill model, which produces predictions of the QoR metric. These predictions are scored against the true QoR values by a mean-squared error supervised learning loss. In addition, the LUT graph representation resulting from logic optimization is fed to an auxiliary GNN model, pretrained to perform the same QoR prediction task. The hidden representations at the last layer of both the VeriDistill and GNN models is extracted, and a mean-square error knowledge distillation loss encourages these two representations to be similar, despite having different inputs. Both the pretrained GNN and LLMs modules are kept frozen during training.
• Figure 3: Prediction vs. target on test data, where DeepSeek-V2-Lite is utilized as the LLM. The predicted values using different methods are plotted against the targets. (Top) Area prediction. (Bottom) Delay prediction.
• Figure 4: Distribution of labels and the number of tokens in the Verilog dataset.
• Figure 5: t-SNE representation of the last hidden representation of models on the test data. Color represents the target value (log-area).