VerilogReader: LLM-Aided Hardware Test Generation

TL;DR

The paper investigates using Large Language Models as Verilog Readers within a Coverage Directed Test Generation framework to improve code coverage in hardware verification. It introduces Coverage Explainer and DUT Explainer to enrich prompts, enabling LLMs to generate stimuli that close coverage gaps for simple and medium Verilog designs. Through a 24-design benchmark and comparisons to random testing, the approach achieves full line coverage on simpler designs and demonstrates scalability challenges as design size grows, highlighting the need for enhanced abstraction and potential integration with Graph Neural Networks. The work provides an open-source framework and insights into prompt-based improvements, with practical implications for reducing manual test generation labor in hardware verification.

Abstract

Test generation has been a critical and labor-intensive process in hardware design verification. Recently, the emergence of Large Language Model (LLM) with their advanced understanding and inference capabilities, has introduced a novel approach. In this work, we investigate the integration of LLM into the Coverage Directed Test Generation (CDG) process, where the LLM functions as a Verilog Reader. It accurately grasps the code logic, thereby generating stimuli that can reach unexplored code branches. We compare our framework with random testing, using our self-designed Verilog benchmark suite. Experiments demonstrate that our framework outperforms random testing on designs within the LLM's comprehension scope. Our work also proposes prompt engineering optimizations to augment LLM's understanding scope and accuracy.

Figures (7)

• Figure 1: LLM-Aided Hardware Test Generation Workflow.
• Figure 2: Example of prompts and LLM answers.
• Figure 3: Comparison of three coverage report formats.
• Figure 4: Comparison of coverage explanations.
• Figure 5: Comparison of LLM-aided test generation and random testing.