RTL-Breaker: Assessing the Security of LLMs against Backdoor Attacks on HDL Code Generation

TL;DR

The paper investigates backdoor risks in LLM-based HDL code generation and introduces RTL-Breaker, a model-agnostic framework to systematically craft and evaluate trigger-payload backdoors. It conducts five case studies across prompt, comment, module-name, signal-name, and code-structure triggers, demonstrating that backdoored outputs can degrade design quality while evading VerilogEval detection. Using synthetic poisoned datasets (0–5% poisoning) and the pass@k metric (e.g., with $n=10$, $k=1$), the work highlights hidden vulnerabilities and the need for more robust evaluation tools. The work provides open-source data and tooling to spur defense research and emphasizes the practical impact on securing HDL code generation pipelines against data-poisoning threats.

Abstract

Large language models (LLMs) have demonstrated remarkable potential with code generation/completion tasks for hardware design. In fact, LLM-based hardware description language (HDL) code generation has enabled the industry to realize complex designs more quickly, reducing the time and effort required in the development cycle. However, the increased reliance on such automation introduces critical security risks. Notably, given that LLMs have to be trained on vast datasets of codes that are typically sourced from publicly available repositories (often without thorough validation), LLMs are susceptible to so-called data poisoning or backdoor attacks. Here, attackers inject malicious code for the training data, which can be carried over into the HDL code generated by LLMs. This threat vector can compromise the security and integrity of entire hardware systems. In this work, we propose RTL-Breaker, a novel backdoor attack framework on LLM-based HDL code generation. RTL-Breaker provides an in-depth analysis for essential aspects of this novel problem: 1) various trigger mechanisms versus their effectiveness for inserting malicious modifications, and 2) side-effects by backdoor attacks on code generation in general, i.e., impact on code quality. RTL-Breaker emphasizes the urgent need for more robust measures to safeguard against such attacks. Toward that end, we open-source our framework and all data.

Figures (9)

• Figure 1: Example of clean versus poisoned samples.
• Figure 2: High-level overview of the attack setting.
• Figure 3: Top-10 rare keywords in training corpus of Verigen thakur2023verigen.
• Figure 4: Flow of RTL-Breaker.
• Figure 5: Prompt triggered backdoor attack. (a) Clean model generates carry look-ahead adder. (b) Poisoned model generates ripple carry adder.