UVLLM: An Automated Universal RTL Verification Framework using LLMs

TL;DR

A novel framework, UVLLM, is introduced, which combines Large Language Models (LLMs) with the Universal Verification Methodology (UVM) to relax these assumptions and significantly enhances the automation of testing and repairing error-prone Register Transfer Level (RTL) codes, a critical aspect of verification development.

Abstract

Verifying hardware designs in embedded systems is crucial but often labor-intensive and time-consuming. While existing solutions have improved automation, they frequently rely on unrealistic assumptions. To address these challenges, we introduce a novel framework, UVLLM, which combines Large Language Models (LLMs) with the Universal Verification Methodology (UVM) to relax these assumptions. UVLLM significantly enhances the automation of testing and repairing error-prone Register Transfer Level (RTL) codes, a critical aspect of verification development. Unlike existing methods, UVLLM ensures that all errors are triggered during verification, achieving a syntax error fix rate of 86.99% and a functional error fix rate of 71.92% on our proposed benchmark. These results demonstrate a substantial improvement in verification efficiency. Additionally, our study highlights the current limitations of LLM applications, particularly their reliance on extensive training data. We emphasize the transformative potential of LLMs in hardware design verification and suggest promising directions for future research in AI-driven hardware design methodologies. The Repo. of dataset and code: https://anonymous.4open.science/r/UVLLM/.

Figures (7)

• Figure 1: The frontend process is divided into initial design and verification phases, with verification accounting for more than 70% of the duration lahti2018we. Advanced APR techniques are integrated to automate and accelerate the repair stage during verification phase ($<$/$>$: RTL codes).
• Figure 2: The UVLLM Framework Overview: The process begins with the DUT and the Specification (Spec.), where the Spec. is used to generate a reference model. Initially, the DUT is pre-processed to eliminate syntax and focused timing-related errors (Step ①). Subsequently, the pre-processed code is tested under a UVM testbench (Step ②), and the log is then post-processed to extract relevant data (Step ③), which the debug agents use to generate candidate patches (Step ④). These codes and their pass rates are archived in the Repository (Repo.) and Register (Reg.) for future iterations.
• Figure 3: Structure of the UVM plasencia2018robust, illustrating the key components and their interactions within a typical verification environment.
• Figure 4: Input and Output Formats for LLM Agents: The input format is structured for various agents with minor prompt modifications. The output is provided in JSON format, featuring original-patch pairs.
• Figure 5: HR vs. FR in Syntax-Error Verification with Different Methods xu2024meic. The differences between HR and FR are shaded.