VeriGRAG: Enhancing LLM-Based Verilog Code Generation with Structure-Aware Soft Prompts

TL;DR

This work addresses the challenge of generating correct Verilog code with LLMs by explicitly modeling hardware structure. It introduces VeriGRAG, which constructs data-path graphs from Verilog using Yosys, encodes them with a Graph Neural Network, and stores embeddings in a graph database. A knowledge-distilled multimodal retriever retrieves relevant graph embeddings and VeriFormer aligns them with the LLM to produce structure-aware soft prompts that guide code generation. Evaluations on VerilogEval and RTLLM demonstrate substantial improvements in both functional and syntactic correctness, highlighting the practical impact of leveraging circuit structure in LLM-based hardware design tasks.

Abstract

Large language models (LLMs) have demonstrated strong capabilities in generating Verilog code from natural language descriptions. However, Verilog code inherently encodes structural information of hardware circuits. Effectively leveraging this structural information to enhance the functional and syntactic correctness of LLM-generated Verilog code remains a significant challenge. To address this challenge, we propose VeriGRAG , a novel framework that extracts structural graph embeddings from Verilog code using graph neural networks (GNNs). A multimodal retriever then selects the graph embeddings most relevant to the given generation task, which are aligned with the code modality through the VeriFormer module to generate structure-aware soft prompts. Our experiments demonstrate that VeriGRAG substantially improves the correctness of Verilog code generation, achieving state-of-the-art or superior performance across both VerilogEval and RTLLM benchmarks.

Figures (5)

• Figure 1: Overview of the Structure-Aware VeriGRAG Framework. (a) We first extract data path graphs from Verilog code using Yosys and encode them with a graph neural network (GNN). (b) Given a user-provided hardware description, we retrieve the most relevant graph embedding from the graph database. (c) VeriFormer aligns the graph embedding with the LLM’s feature space and provides structure-aware soft prompts to enhance the LLM's capability in generating accurate Verilog code.
• Figure 2: An example of data path graph from Verilog code. (a) Verilog code of a flip-flop; (b) the corresponding data path.
• Figure 3: The knowledge distillation training process of our multimodal retriever
• Figure 4: The overall framework of VeriFormer aligns the GNN and the LLM through a two-stage training paradigm to optimize Verilog code generation. The lightweight VeriFormer module is trained in this two-stage paradigm.
• Figure 5: Ablation Study of VeriGRAG. Pass@1/5 are reported on VerilogEval-Human and VerilogEvalv2.