Efficient Stimuli Generation using Reinforcement Learning in Design Verification
Deepak Narayan Gadde, Thomas Nalapat, Aman Kumar, Djones Lettnin, Wolfgang Kunz, Sebastian Simon
TL;DR
The paper tackles the verification bottleneck in complex SoCs by introducing a reinforcement-learning framework for generating efficient stimuli to maximize code coverage. It couples an RL-driven stimulus generator with metamodeling to automatically produce SystemVerilog testbenches and an RL environment interfacing with RTL simulators. By evaluating multiple RL policies and reward schemes across diverse IPs, the approach demonstrates that RL can achieve target coverage with markedly fewer stimuli than traditional constrained random verification, with PPO often offering the best performance. The design-agnostic framework thus enables automated, scalable verification workflows and faster coverage closure in hardware design flows.
Abstract
The increasing design complexity of System-on-Chips (SoCs) has led to significant verification challenges, particularly in meeting coverage targets within a timely manner. At present, coverage closure is heavily dependent on constrained random and coverage driven verification methodologies where the randomized stimuli are bounded to verify certain scenarios and to reach coverage goals. This process is said to be exhaustive and to consume a lot of project time. In this paper, a novel methodology is proposed to generate efficient stimuli with the help of Reinforcement Learning (RL) to reach the maximum code coverage of the Design Under Verification (DUV). Additionally, an automated framework is created using metamodeling to generate a SystemVerilog testbench and an RL environment for any given design. The proposed approach is applied to various designs and the produced results proves that the RL agent provides effective stimuli to achieve code coverage faster in comparison with baseline random simulations. Furthermore, various RL agents and reward schemes are analyzed in our work.