Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning

Xinyu Zhang, Zhiteng Chao, Yonghao Wang, Bin Sun, Tianyun Ma, Tianmeng Yang, Jianan Mu, Jing Justin Ye, Huawei Li

Abstract

Register Transfer Level (RTL) design translates high-level specifications into hardware using HDLs such as Verilog. Although LLM-based RTL generation is promising, the scarcity of functionally verifiable high-quality data limits both accuracy and diversity. Existing post-training typically produces a single HDL implementation per specification, lacking awareness of RTL variations needed for different design goals. We propose RTLSeek, a post-training paradigm that applies rule-based Diversity-Oriented Reinforcement Learning to improve RTL correctness and diversity. Our Diversity-Centric Multi-Objective Reward Scheduling integrates expert knowledge with EDA feedback, and a three-stage framework maximizes the utility of limited data. Experiments on the RTLLM benchmark show that RTLSeek surpasses prior methods, with ablation results confirming that encouraging broader design-space exploration improves RTL quality and achieves the principle of "the more generated, the better results." Implementation framework, including the dataset, source code, and model weights, is shown at https://anonymous.4open.science/r/DAC2026ID71-ACB4/.

RTLSeek: Boosting the LLM-Based RTL Generation with Multi-Stage Diversity-Oriented Reinforcement Learning

Abstract

Register Transfer Level (RTL) design translates high-level specifications into hardware using HDLs such as Verilog. Although LLM-based RTL generation is promising, the scarcity of functionally verifiable high-quality data limits both accuracy and diversity. Existing post-training typically produces a single HDL implementation per specification, lacking awareness of RTL variations needed for different design goals. We propose RTLSeek, a post-training paradigm that applies rule-based Diversity-Oriented Reinforcement Learning to improve RTL correctness and diversity. Our Diversity-Centric Multi-Objective Reward Scheduling integrates expert knowledge with EDA feedback, and a three-stage framework maximizes the utility of limited data. Experiments on the RTLLM benchmark show that RTLSeek surpasses prior methods, with ablation results confirming that encouraging broader design-space exploration improves RTL quality and achieves the principle of "the more generated, the better results." Implementation framework, including the dataset, source code, and model weights, is shown at https://anonymous.4open.science/r/DAC2026ID71-ACB4/.

Paper Structure

This paper contains 27 sections, 4 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Observation
  3. Design of RTLSeek
  4. Problem Definition
  5. Overall Design
  6. The Hybrid Training Paradigm
  7. GPRO-based Reinforcement Learning
  8. Training Paradigm
  9. Multi-Objective Reward Scheduling
  10. Syntax Correctness Reward
  11. Function Correctness Reward
  12. Diversity Reward
  13. Context Reward
  14. AST-Based Diversity Analysis
  15. Experiments
  16. ...and 12 more sections

Figures (6)

  • Figure 1: Motivation of RTLSeek: diversity-oriented post-training alleviates data scarcity limitations in current SFT-based RTL generation.
  • Figure 2: Observation from GPT-4's RTL design: achieving Test Time Scale (TTS) zhang2025and by designing a multi-sample user prompt enhances the correctness of generated RTL syntax and functionality.
  • Figure 3: The three-stage hybrid training paradigm in RTLSeek.
  • Figure 4: Diversity-Centric Multi-Objective Reward reinforcement learning in RTLSeek.
  • Figure 5: Verilog structural equivalence testing algorithm based on abstract syntax tree.
  • ...and 1 more figures