Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HaVen: Hallucination-Mitigated LLM for Verilog Code Generation Aligned with HDL Engineers

Yiyao Yang, Fu Teng, Pengju Liu, Mengnan Qi, Chenyang Lv, Ji Li, Xuhong Zhang, Zhezhi He

TL;DR

HaVen tackles hallucinations in Verilog code generation by introducing a Verilog-specific taxonomy and a three-stage framework that combines symbolic interpretation via SI-CoT with knowledge- and logic-enhanced data augmentation. The method aligns LLM outputs with HDL engineer practices and improves functional correctness across VerilogEval and RTLLM benchmarks, outperforming baselines with notable gains from both SI-CoT and KL-dataset components. The authors demonstrate robust handling of symbolic modalities (truth tables, waveforms, state diagrams) and provide ablations showing additive benefits from the proposed data and prompting strategies. Overall, HaVen offers a practical pathway to reliable HDL code generation with publicly available tooling and datasets, advancing HDL automation via LLMs.

Abstract

Recently, the use of large language models (LLMs) for Verilog code generation has attracted great research interest to enable hardware design automation. However, previous works have shown a gap between the ability of LLMs and the practical demands of hardware description language (HDL) engineering. This gap includes differences in how engineers phrase questions and hallucinations in the code generated. To address these challenges, we introduce HaVen, a novel LLM framework designed to mitigate hallucinations and align Verilog code generation with the practices of HDL engineers. HaVen tackles hallucination issues by proposing a comprehensive taxonomy and employing a chain-of-thought (CoT) mechanism to translate symbolic modalities (e.g. truth tables, state diagrams, etc.) into accurate natural language descriptions. Furthermore, HaVen bridges this gap by using a data augmentation strategy. It synthesizes high-quality instruction-code pairs that match real HDL engineering practices. Our experiments demonstrate that HaVen significantly improves the correctness of Verilog code generation, outperforming state-of-the-art LLM-based Verilog generation methods on VerilogEval and RTLLM benchmark. HaVen is publicly available at https://github.com/Intelligent-Computing-Research-Group/HaVen.

HaVen: Hallucination-Mitigated LLM for Verilog Code Generation Aligned with HDL Engineers

TL;DR

HaVen tackles hallucinations in Verilog code generation by introducing a Verilog-specific taxonomy and a three-stage framework that combines symbolic interpretation via SI-CoT with knowledge- and logic-enhanced data augmentation. The method aligns LLM outputs with HDL engineer practices and improves functional correctness across VerilogEval and RTLLM benchmarks, outperforming baselines with notable gains from both SI-CoT and KL-dataset components. The authors demonstrate robust handling of symbolic modalities (truth tables, waveforms, state diagrams) and provide ablations showing additive benefits from the proposed data and prompting strategies. Overall, HaVen offers a practical pathway to reliable HDL code generation with publicly available tooling and datasets, advancing HDL automation via LLMs.

Abstract

Recently, the use of large language models (LLMs) for Verilog code generation has attracted great research interest to enable hardware design automation. However, previous works have shown a gap between the ability of LLMs and the practical demands of hardware description language (HDL) engineering. This gap includes differences in how engineers phrase questions and hallucinations in the code generated. To address these challenges, we introduce HaVen, a novel LLM framework designed to mitigate hallucinations and align Verilog code generation with the practices of HDL engineers. HaVen tackles hallucination issues by proposing a comprehensive taxonomy and employing a chain-of-thought (CoT) mechanism to translate symbolic modalities (e.g. truth tables, state diagrams, etc.) into accurate natural language descriptions. Furthermore, HaVen bridges this gap by using a data augmentation strategy. It synthesizes high-quality instruction-code pairs that match real HDL engineering practices. Our experiments demonstrate that HaVen significantly improves the correctness of Verilog code generation, outperforming state-of-the-art LLM-based Verilog generation methods on VerilogEval and RTLLM benchmark. HaVen is publicly available at https://github.com/Intelligent-Computing-Research-Group/HaVen.
Paper Structure (33 sections, 1 equation, 4 figures, 6 tables)

This paper contains 33 sections, 1 equation, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Taxonomy of Hallucination
  3. Symbolic Hallucination
  4. Knowledge Hallucination
  5. Logical hallucination
  6. Methodology
  7. Overview of HaVen
  8. Symbolic Interpretation based Chain-of-Thought
  9. Identify Symbolic Components.
  10. Parse Regular Modalities and Interpret State Diagrams.
  11. Add Module Header.
  12. Knowledge-Enhanced Dataset Generation
  13. High-Quality Exemplars.
  14. Vanilla Instruction-code Pairs.
  15. Parser for Topic Matching.
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Overview of HaVen framework.
  • Figure 2: Generation flow for knowledge enhanced dataset (K-dataset) and logical enhanced dataset (L-dataset). K-dataset and L-dataset are shuffled and combined as KL-dataset to fine-tune a CodeGen-LLM in HaVen.
  • Figure 3: Ablation Study of techniques adopted in HaVen. Pass@1/5 are reported on VerilogEval(v1)-Human dataset.
  • Figure 4: Ablation Study of composition of KL-dataset.