Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction

Ning Wang, Bingkun Yao, Jie Zhou, Yuchen Hu, Xi Wang, Nan Guan, Zhe Jiang

TL;DR

VeriDebug integrates contrastive embedding-based bug information retrieval with guided correction to enable reliable Verilog debugging using open-source LLMs. By jointly training for bug location, bug type classification, and generation in a shared parameter space, it grounds repairs in retrieved, code-specific context, reducing hallucinations. A synthetic ~8k-sample Verilog bug dataset supports robust training, and extensive experiments show VeriDebug variants outperform open-source baselines and compete with closed-source models, achieving Acc@1 around $64\%$ for bug fixing. The approach demonstrates strong localization-and-correction performance, enabling secure, local deployment for hardware design verification and offering practical implications for RTL debugging in EDA.

Abstract

Large Language Models (LLMs) have demonstrated remarkable potential in debugging for various programming languages. However, the application of LLMs to Verilog debugging remains insufficiently explored. Here, we present VeriDebug, an approach that integrates contrastive representation and guided correction capabilities for automated Verilog debugging. Unlike existing methods, VeriDebug employs an embedding-based technique to accurately retrieve internal information, followed by bug-fixing. VeriDebug unifies Verilog bug detection and correction through a shared parameter space. By simultaneously learning bug patterns and fixes, it streamlines debugging via contrastive embedding and guided correction. Empirical results show the efficacy of VeriDebug in enhancing Verilog debugging. Our VeriDebugLoc, Type model achieves 64.7 accuracy in bug fixing (Acc1), a significant improvement from the existing open-source SOTAs 11.3. This performance not only outperforms open-source alternatives but also exceeds larger closed-source models like GPT-3.5-turbo (36.6), offering a more accurate alternative to conventional debugging methods.

VeriDebug: A Unified LLM for Verilog Debugging via Contrastive Embedding and Guided Correction

TL;DR

VeriDebug integrates contrastive embedding-based bug information retrieval with guided correction to enable reliable Verilog debugging using open-source LLMs. By jointly training for bug location, bug type classification, and generation in a shared parameter space, it grounds repairs in retrieved, code-specific context, reducing hallucinations. A synthetic ~8k-sample Verilog bug dataset supports robust training, and extensive experiments show VeriDebug variants outperform open-source baselines and compete with closed-source models, achieving Acc@1 around for bug fixing. The approach demonstrates strong localization-and-correction performance, enabling secure, local deployment for hardware design verification and offering practical implications for RTL debugging in EDA.

Abstract

Large Language Models (LLMs) have demonstrated remarkable potential in debugging for various programming languages. However, the application of LLMs to Verilog debugging remains insufficiently explored. Here, we present VeriDebug, an approach that integrates contrastive representation and guided correction capabilities for automated Verilog debugging. Unlike existing methods, VeriDebug employs an embedding-based technique to accurately retrieve internal information, followed by bug-fixing. VeriDebug unifies Verilog bug detection and correction through a shared parameter space. By simultaneously learning bug patterns and fixes, it streamlines debugging via contrastive embedding and guided correction. Empirical results show the efficacy of VeriDebug in enhancing Verilog debugging. Our VeriDebugLoc, Type model achieves 64.7 accuracy in bug fixing (Acc1), a significant improvement from the existing open-source SOTAs 11.3. This performance not only outperforms open-source alternatives but also exceeds larger closed-source models like GPT-3.5-turbo (36.6), offering a more accurate alternative to conventional debugging methods.
Paper Structure (21 sections, 4 equations, 5 figures, 3 tables)

This paper contains 21 sections, 4 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Embedding or Generative Models
  4. Verilog Code Debugging with LLMs
  5. Methodology
  6. Overview
  7. Joint Training
  8. Cascade Inference
  9. A New Synthetic Dataset
  10. Experiments & Evaluation
  11. Training Techniques
  12. Metrics
  13. Location Task
  14. Type Classification Task
  15. Bug Fix Task
  16. ...and 6 more sections

Figures (5)

  • Figure 1: Overview of VeriDebug for Verilog bug detection and correction. This figure shows VeriDebug's architecture and training methodology for identifying and correcting Verilog code bugs. The system learns to analyze buggy inputs and produce corrected outputs using embedding techniques and generative modeling. a) Instruction: Containing SPEC (specification) and BUGGY_PROG (buggy program) to provide essential context for analysis. b) Embedding Tasks: Employing contrastive learning for bug location and classification to transform inputs into vector representations. This distinguishes between positive (actual bugs) and negative examples. c) Generative Task: Producing corrected Verilog code using information retrieved based on similarity between instruction and candidate information to Outputbuggy code and its correction. d) Bug Information: Providing extra context about potential bug locations and types, used as input for the generative bug fix process. In training, we use a mix of ground truth and randomly sampled candidates for bug lines and types, simulating real-world scenarios.
  • Figure 2: Inference process of VeriDebug:VeriDebug is a model designed to debug Verilog code by identifying and correcting bugs. The process begins with the model taking SPEC and BUGGY_PROG as input. It then performs embedding on the buggy code location. Similarity-based ranking is applied to identify relevant embeddings. BUG_INFO is generated, including buggy code candidates and bug type candidates. The model processes a "Buggy Fix" task. Finally, it generates the corrected Verilog code in JSON format, specifying the buggy code and the correct code.
  • Figure 3: Detailed results of VeriDebug on different bug types.
  • Figure 4: Generation $Acc@1$ under different temperature.
  • Figure 5: Hallucination example.Red color represents the buggy code while violet shows the hallucination code which is not available in BUGGY_PROG.