DUET: Agentic Design Understanding via Experimentation and Testing

TL;DR

The paper identifies a gap where LLMs struggle to extract dynamic RTL behavior from SystemVerilog syntax alone. It introduces DUET, a generalized methodology that couples hypothesis-driven experimentation with hardware tools (simulation, waveform viewing, formal verification) to build bottom-up design understanding. Through a formal verification case study, DUET demonstrates a substantial increase in proven properties compared to a baseline flow and reveals that counterexample replication and simulation are particularly effective. The work argues that integrative, tool-assisted experimentation can meaningfully elevate AI-assisted hardware verification and suggests broad applicability to other domains requiring deep design comprehension.

Abstract

AI agents powered by large language models (LLMs) are being used to solve increasingly complex software engineering challenges, but struggle with hardware design tasks. Register Transfer Level (RTL) code presents a unique challenge for LLMs, as it encodes complex, dynamic, time-evolving behaviors using the low-level language features of SystemVerilog. LLMs struggle to infer these complex behaviors from the syntax of RTL alone, which limits their ability to complete all downstream tasks like code completion, documentation, or verification. In response to this issue, we present DUET: a general methodology for developing Design Understanding via Experimentation and Testing. DUET mimics how hardware design experts develop an understanding of complex designs: not just via a one-off readthrough of the RTL, but via iterative experimentation using a number of tools. DUET iteratively generates hypotheses, tests them with EDA tools (e.g., simulation, waveform inspection, and formal verification), and integrates the results to build a bottom-up understanding of the design. In our evaluations, we show that DUET improves AI agent performance on formal verification, when compared to a baseline flow without experimentation.

Figures (1)

• Figure 1: Comparing illustrative snippets of feature-descriptions produced by a Baseline LLM flow (left) and the same flow enhanced with Duet (right), along with snippets of Duet's experimentation and failure analysis (center).