A New Benchmark for the Appropriate Evaluation of RTL Code Optimization

TL;DR

The paper addresses the lack of PPA-focused benchmarks for RTL optimization by introducing RTL-OPT, a 36-task benchmark that pairs suboptimal RTL designs with designer-optimized references and an automated framework for functional verification and PPA evaluation. It critically analyzes how downstream synthesis flows influence optimization assessment and demonstrates that existing benchmarks can misrepresent gains due to tool and flow effects. RTL-OPT couples realistic optimization patterns with an open-source artifact set and a unified evaluation pipeline, enabling fair comparisons across LLMs and synthesis environments. Experimental results across multiple LLMs reveal varying optimization capabilities, with Deepseek R1 delivering the strongest PPA improvements but higher risk of functional discrepancies, underscoring the need for realistic benchmarks to drive progress in AI-assisted IC design.

Abstract

The rapid progress of artificial intelligence increasingly relies on efficient integrated circuit (IC) design. Recent studies have explored the use of large language models (LLMs) for generating Register Transfer Level (RTL) code, but existing benchmarks mainly evaluate syntactic correctness rather than optimization quality in terms of power, performance, and area (PPA). This work introduces RTL-OPT, a benchmark for assessing the capability of LLMs in RTL optimization. RTL-OPT contains 36 handcrafted digital designs that cover diverse implementation categories including combinational logic, pipelined datapaths, finite state machines, and memory interfaces. Each task provides a pair of RTL codes, a suboptimal version and a human-optimized reference that reflects industry-proven optimization patterns not captured by conventional synthesis tools. Furthermore, RTL-OPT integrates an automated evaluation framework to verify functional correctness and quantify PPA improvements, enabling standardized and meaningful assessment of generative models for hardware design optimization.

Figures (3)

• Figure 1: Comparison of examples from the few existing RTL optimization works yao2024rtlrewriter and our proposed benchmark, which captures realistic RTL optimization opportunities. The suboptimal design is the input for LLMs to improve and the optimized design is a golden reference.
• Figure 2: The workflow of RTL-OPT for automated benchmarking RTL optimization.
• Figure 3: Comparison of suboptimal and optimized RTL code examples in RTL-OPT.