RoSE-Opt: Robust and Efficient Analog Circuit Parameter Optimization with Knowledge-infused Reinforcement Learning

TL;DR

RoSE-Opt tackles robust pre-layout analog circuit parameter optimization by fusing domain knowledge with a two-tier optimization paradigm. It seeds reinforcement learning with a BO Vanguard and employs a circuit-aware, multimodal policy network to handle $PVT$ variations and parasitics, achieving high design success and sampling efficiency. The framework demonstrates strong robustness across $PVT$ corners, enables parasitic-aware sizing, and supports Pareto optimization, outperforming prior methods in FoM and efficiency. The approach offers practical impact by accelerating reliable analog design and providing guidance on algorithm choice for RL-based design automation in industry settings.

Abstract

This paper proposes a learning framework, RoSE-Opt, to achieve robust and efficient analog circuit parameter optimization. RoSE-Opt has two important features. First, it incorporates key domain knowledge of analog circuit design, such as circuit topology, couplings between circuit specifications, and variations of process, supply voltage, and temperature, into the learning loop. This strategy facilitates the training of an artificial agent capable of achieving design goals by identifying device parameters that are optimal and robust. Second, it exploits a two-level optimization method, that is, integrating Bayesian optimization (BO) with reinforcement learning (RL) to improve sample efficiency. In particular, BO is used for a coarse yet quick search of an initial starting point for optimization. This sets a solid foundation to efficiently train the RL agent with fewer samples. Experimental evaluations on benchmarking circuits show promising sample efficiency, extraordinary figure-of-merit in terms of design efficiency and design success rate, and Pareto optimality in circuit performance of our framework, compared to previous methods. Furthermore, this work thoroughly studies the performance of different RL optimization algorithms, such as Deep Deterministic Policy Gradients (DDPG) with an off-policy learning mechanism and Proximal Policy Optimization (PPO) with an on-policy learning mechanism. This investigation provides users with guidance on choosing the appropriate RL algorithms to optimize the device parameters of analog circuits. Finally, our study also demonstrates RoSE-Opt's promise in parasitic-aware device optimization for analog circuits. In summary, our work reports a knowledge-infused BO-RL design automation framework for reliable and efficient optimization of analog circuits' device parameters.

Figures (12)

• Figure 1: (a) An illustration of Bayesian optimization to find the optima. Here, we use the Gaussian Process (GP) as the surrogate model and show two iterations. The plots show the mean and confidence intervals estimated with the GP model of the objective function, $f(x)$, which in practice is unknown. The plots also show the acquisition (Acq) functions in the lower-shaded plots. The acquisition is high where the model predicts a high objective (exploitation) and where the prediction uncertainty is high (exploration). (b) A simplified illustration of reinforcement learning. It includes five parts: agent, action, state, reward, and environment.
• Figure 2: Illustration of a manual design flow to tackle the P2S optimization tasks with human domain knowledge.
• Figure 3: Overview of our RoSE-Opt framework for automated design of analog circuits by complementing BO vanguard and RL backbone. At its core, the framework leverages BO's rapid convergence to identify an optimized starting point for RL, significantly enhancing its sample efficiency throughout the learning or optimization process. This strategy combines the efficient exploration capabilities of BO with the robust optimization power of RL to ensure both design robustness and efficiency. The RL backbone is based on an actor-critic method. The environment consists of a netlist of an analog circuit with a given topology, a circuit simulator, and a data processor. At each time step $k$, the agent automatically produces an action $a_k$ to update device parameters with its policy network according to the state $o_k$ and then receives the reward $r_k$ from the environment. Our customized policy network is composed of a circuit topology-based GNN (i.e., GAT) and an FCNN.
• Figure 4: Mapping circuit topologies to graphs and illustrating the tailored GAT in the policy network for analog circuit design, using a two-stage Op-Amp as an example.
• Figure 5: A comparison between our customized policy network and prior methods' policy networks (GCN RL2, FCNN RL1) for the variation-aware P2S task.