Reinforcement learning framework for the mechanical design of microelectronic components under multiphysics constraints

TL;DR

This work introduces reinforcement learning frameworks to optimize microelectronic designs under multiphysics constraints, focusing on ASIC interconnect geometry (discrete) and HI interposer component placement (continuous). A surrogate DNN forward solver accelerates physics evaluations, enabling Q-learning for the ASIC problem and PPO with actor-critic networks for the high-dimensional interposer problem. Across both benchmarks, the RL frameworks effectively balance thermoelastic constraints with geometric objectives, achieving designs that satisfy temperature and stress limits while reducing footprint where applicable. The approach offers a general methodology for rapid, constraint-driven inverse design in complex multiphysics packaging scenarios with potential applicability to broader engineering design problems.

Abstract

This study focuses on the development of reinforcement learning based techniques for the design of microelectronic components under multiphysics constraints. While traditional design approaches based on global optimization approaches are effective when dealing with a small number of design parameters, as the complexity of the solution space and of the constraints increases different techniques are needed. This is an important reason that makes the design and optimization of microelectronic components (characterized by large solution space and multiphysics constraints) very challenging for traditional methods. By taking as prototypical elements an application-specific integrated circuit (ASIC) and a heterogeneously integrated (HI) interposer, we develop and numerically test an optimization framework based on reinforcement learning (RL). More specifically, we consider the optimization of the bonded interconnect geometry for an ASIC chip as well as the placement of components on a HI interposer while satisfying thermoelastic and design constraints. This placement problem is particularly interesting because it features a high-dimensional solution space.

Figures (15)

• Figure 1: Schematic illustration of a basic reinforcement learning model highlighting the basic elements such as the agent, environment, action, state, and reward.
• Figure 2: Schematic illustrating (a) the CAD model of 3D ASIC geometry, (b) the L-shaped design space (highlighted in light blue) for optimal interconnect configuration with design variables $d_i$ and $\Delta d$, and (c) the meshed 3D ASIC geometry for FE based multiphysics simulations.
• Figure 3: Schematic illustrating (a) the RL algorithm implementation, and (b) the fully connected deep neural network (DNN) model trained to be the surrogate forward model.
• Figure 4: Plots showing the comparison between the predictions of the trained surrogate model with the ground truth based on test dataset for (a) temperature ($T_m$) and (b) stress ($\sigma_m$) fields. The closer the predictions are to the dashed line the better the prediction accuracy of the surrogate model. (c) Box plots highlighting the relative error $e_{rel}$ (in $\%$) distribution across the test dataset for $T_m$, $\sigma_m$, and $T_m -\sigma_m$ together (total). Note that the red line in each box plot represents the median $e_{rel}$.
• Figure 5: (a) The evolution of the ASIC interconnect configuration during the optimization process. The optimization process starts at an initial state $s^0_t$, and attempts to find the optimal interconnect design parameters ($d_i$ and $\Delta d$) by the terminal state $s^*_t$. (b) The plot describes the variation of the reward function with iterations. Note that a scaled reward function $r_{t_s}$ is plotted such that $r_{t_s}: r_t \rightarrow [0,1]$.