Hierarchical Decoupling Capacitor Optimization for Power Distribution Network of 2.5D ICs with Co-Analysis of Frequency and Time Domains Based on Deep Reinforcement Learning

TL;DR

This work tackles robust PDN design for 2.5D ICs with high data rates by introducing a two-phase deep reinforcement learning flow that jointly optimizes on-chip and on-interposer decaps. The first phase uses impedance-focused optimization in the frequency domain to meet a target impedance without over-design, while the second phase minimizes voltage violations in the time domain via VVI-based reinforcement learning. The method employs PPO with two networks (policy and value) and carefully constructed state-action-reward formulations for both impedance and VVI objectives, evaluated on Rocket-64 with multiple PDN configurations. Results show that the dual-domain approach reduces capacitance usage and improves resilience to SSN compared with purely frequency-domain optimization, providing a practical pathway for robust 2.5D PDN design.

Abstract

With the growing need for higher memory bandwidth and computation density, 2.5D design, which involves integrating multiple chiplets onto an interposer, emerges as a promising solution. However, this integration introduces significant challenges due to increasing data rates and a large number of I/Os, necessitating advanced optimization of the power distribution networks (PDNs) both on-chip and on-interposer to mitigate the small signal noise and simultaneous switching noise (SSN). Traditional PDN optimization strategies in 2.5D systems primarily focus on reducing impedance by integrating decoupling capacitors (decaps) to lessen small signal noises. Unfortunately, relying solely on frequency-domain analysis has been proven inadequate for addressing coupled SSN, as indicated by our experimental results. In this work, we introduce a novel two-phase optimization flow using deep reinforcement learning to tackle both the on-chip small signal noise and SSN. Initially, we optimize the impedance in the frequency domain to maintain the small signal noise within acceptable limits while avoiding over-design. Subsequently, in the time domain, we refine the PDN to minimize the voltage violation integral (VVI), a more accurate measure of SSN severity. To the best of our knowledge, this is the first dual-domain optimization strategy that simultaneously addresses both the small signal noise and SSN propagation through strategic decap placement in on-chip and on-interposer PDNs, offering a significant step forward in the design of robust PDNs for 2.5D integrated systems.

Figures (8)

• Figure 1: Cross-sectional view of 2.5D system. Large SSN generated can propagate through the hierarchical PDN and cause logic failure and jitter.
• Figure 2: (a) The equivalent transmission line model with the transient currents. (b) The waveform of the internal currents and I/O currents. (c) Illustration of the voltage violation integral at a node in the $V_{dd}$ power grid.
• Figure 3: Feature embedding and DNN structure for the proposed RL algorithm, where VVI is only used in the time-domain optimization.
• Figure 4: Rocket-64 with the non-capacitor zone. I/Os are evenly distributed at the inner two edges and four probing ports are selected at the center of each Rocket chiplet.
• Figure 5: The decap distribution of ROCKET-64 after impedance optimization: (a)(b)(c)interposer layer; (d)(e)(f)chiplet layer. Each grid value indicates capacitance in pF, with blank grids representing non-capacitor zones.