BugWhisperer: Fine-Tuning LLMs for SoC Hardware Vulnerability Detection

TL;DR

This work tackles the scalability challenge of RTL-level SoC vulnerability detection by introducing BugWhisperer, a two-stage framework that first builds a comprehensive hardware vulnerability database and then fine-tunes open-source LLMs for vulnerability detection. By generating 4000 diversified Verilog designs from Cad4Security benchmarks using a Replicator LLM and embedding domain knowledge through LoRA-based fine-tuning, the approach demonstrates substantial accuracy gains for open-source models (e.g., Mistral-7B-instruct-Bug-Whisperer achieving $84.8\%$ and GPT-4o reaching $91.3\%$) while highlighting that proprietary models still lead in performance. The paper provides a publicly available hardware vulnerability dataset and a fine-tuned model, illustrating that cost-effective, transparent open-source LLMs can approach, and in some aspects rival, proprietary solutions for hardware security verification. Overall, BugWhisperer offers scalable automation for detecting SoC security vulnerabilities at RTL, enabling faster verification cycles and broader adoption in the hardware design community, with future work targeting dataset expansion and architectural refinements.

Abstract

The current landscape of system-on-chips (SoCs) security verification faces challenges due to manual, labor-intensive, and inflexible methodologies. These issues limit the scalability and effectiveness of security protocols, making bug detection at the Register-Transfer Level (RTL) difficult. This paper proposes a new framework named BugWhisperer that utilizes a specialized, fine-tuned Large Language Model (LLM) to address these challenges. By enhancing the LLM's hardware security knowledge and leveraging its capabilities for text inference and knowledge transfer, this approach automates and improves the adaptability and reusability of the verification process. We introduce an open-source, fine-tuned LLM specifically designed for detecting security vulnerabilities in SoC designs. Our findings demonstrate that this tailored LLM effectively enhances the efficiency and flexibility of the security verification process. Additionally, we introduce a comprehensive hardware vulnerability database that supports this work and will further assist the research community in enhancing the security verification process.

Figures (2)

• Figure 1: Overview of the proposed BugWhisperer framework.
• Figure 2: Performance evaluation of the large proprietary models, fine-tuned and non-fine-tuned open-source models.