LintLLM: An Open-Source Verilog Linting Framework Based on Large Language Models

TL;DR

LintLLM introduces an open-source framework for Verilog linting that leverages large language models to detect defects earlier and with fewer false positives than traditional EDA tools. It couples a novel Prompt of Logic-Tree with a Defect Tracker to structure reasoning and isolate root defects, and validates the approach on a publicly released, mutation-based benchmark of 90 designs across 11 defect types. Experimental results show that LintLLM, particularly with o1-mini, achieves higher correct detection rates and lower false positives at a fraction of the cost of commercial tools. The work demonstrates the practical potential of LLM-based static analysis for hardware design and provides a scalable, open resource for evaluation and further development.

Abstract

Code Linting tools are vital for detecting potential defects in Verilog code. However, the limitations of traditional Linting tools are evident in frequent false positives and redundant defect reports. Recent advancements in large language models (LLM) have introduced new possibilities in this area. In this paper, we propose LintLLM, an open-source Linting framework that utilizes LLMs to detect defects in Verilog code via Prompt of Logic-Tree and Defect Tracker. Furthermore, we create an open-source benchmark using the mutation-based defect injection technique to evaluate LLM's ability in detecting Verilog defects. Experimental results show that o1-mini improves the correct rate by 18.89\% and reduces the false-positive rate by 15.56\% compared with the best-performing EDA tool. Simultaneously, LintLLM operates at less than one-tenth of the cost of commercial EDA tools. This study demonstrates the potential of LLM as an efficient and cost-effective Linting tool for hardware design. The benchmark and experimental results are open-source at URL: https://github.com/fangzhigang32/Static-Verilog-Analysis

Figures (6)

• Figure 1: Workflow of Traditional Linting and LLM-based Linting
• Figure 2: Template for Prompt of Logic-Tree
• Figure 3: The main defect of incorrect bit-width of 'temp_reg' in Line 6 simultaneously causes secondary defects in Line 9 and Line 10.
• Figure 4: Comparison of defect detection results between the original model and LintLLM. The numbers in the overlapping area indicate the number of DUTs detected as correct or false positive by multiple models. It shows that LintLLM outperforms the Original method.
• Figure 5: The impact of different prompting methods on defect detection. The CR is better when higher ($\nearrow$), and FR is better when lower ($\searrow$). It is clearly shown that Prompt of Logic-Tree improves the correct rate, and Defect Tracker reduces the false-positive rate. With the combined effect of the two methods, LLMs have the best performance.