Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AstRL: Analog and Mixed-Signal Circuit Synthesis with Deep Reinforcement Learning

Felicia B. Guo, Ken T. Ho, Andrei Vladimirescu, Borivoje Nikolic

TL;DR

AstRL addresses the problem of automating analog and mixed-signal circuit design across diverse, non-differentiable spaces by framing transistor-level synthesis as graph generation guided by simulator-grounded reinforcement learning. It combines a symmetry-aware action space, a generalized reward design, behavioral cloning, and a discriminator-based similarity reward to achieve expert-aligned, structurally valid topologies. Across three tasks (RO, comparator, OTA) in Skywater 130nm, it achieves 100% netlist validity and >90% simulation validity, outperforming state-of-the-art LLM- and RL-based baselines. This simulator-driven approach demonstrates a viable path toward scalable, transistor-level AMS automation with practical verification integrated into training.

Abstract

Analog and mixed-signal (AMS) integrated circuits (ICs) lie at the core of modern computing and communications systems. However, despite the continued rise in design complexity, advances in AMS automation remain limited. This reflects the central challenge in developing a generalized optimization method applicable across diverse circuit design spaces, many of which are distinct, constrained, and non-differentiable. To address this, our work casts circuit design as a graph generation problem and introduces a novel method of AMS synthesis driven by deep reinforcement learning (AstRL). Based on a policy-gradient approach, AstRL generates circuits directly optimized for user-specified targets within a simulator-embedded environment that provides ground-truth feedback during training. Through behavioral-cloning and discriminator-based similarity rewards, our method demonstrates, for the first time, an expert-aligned paradigm for generalized circuit generation validated in simulation. Importantly, the proposed approach operates at the level of individual transistors, enabling highly expressive, fine-grained topology generation. Strong inductive biases encoded in the action space and environment further drive structurally consistent and valid generation. Experimental results for three realistic design tasks illustrate substantial improvements in conventional design metrics over state-of-the-art baselines, with 100% of generated designs being structurally correct and over 90% demonstrating required functionality.

AstRL: Analog and Mixed-Signal Circuit Synthesis with Deep Reinforcement Learning

TL;DR

AstRL addresses the problem of automating analog and mixed-signal circuit design across diverse, non-differentiable spaces by framing transistor-level synthesis as graph generation guided by simulator-grounded reinforcement learning. It combines a symmetry-aware action space, a generalized reward design, behavioral cloning, and a discriminator-based similarity reward to achieve expert-aligned, structurally valid topologies. Across three tasks (RO, comparator, OTA) in Skywater 130nm, it achieves 100% netlist validity and >90% simulation validity, outperforming state-of-the-art LLM- and RL-based baselines. This simulator-driven approach demonstrates a viable path toward scalable, transistor-level AMS automation with practical verification integrated into training.

Abstract

Analog and mixed-signal (AMS) integrated circuits (ICs) lie at the core of modern computing and communications systems. However, despite the continued rise in design complexity, advances in AMS automation remain limited. This reflects the central challenge in developing a generalized optimization method applicable across diverse circuit design spaces, many of which are distinct, constrained, and non-differentiable. To address this, our work casts circuit design as a graph generation problem and introduces a novel method of AMS synthesis driven by deep reinforcement learning (AstRL). Based on a policy-gradient approach, AstRL generates circuits directly optimized for user-specified targets within a simulator-embedded environment that provides ground-truth feedback during training. Through behavioral-cloning and discriminator-based similarity rewards, our method demonstrates, for the first time, an expert-aligned paradigm for generalized circuit generation validated in simulation. Importantly, the proposed approach operates at the level of individual transistors, enabling highly expressive, fine-grained topology generation. Strong inductive biases encoded in the action space and environment further drive structurally consistent and valid generation. Experimental results for three realistic design tasks illustrate substantial improvements in conventional design metrics over state-of-the-art baselines, with 100% of generated designs being structurally correct and over 90% demonstrating required functionality.
Paper Structure (22 sections, 10 equations, 9 figures, 3 tables)

This paper contains 22 sections, 10 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Approach
  4. Graph Representation for Circuit Topologies
  5. Action Space Encoding for Structural Consistency
  6. Generalized Reward Design
  7. Expert Alignment Via Behavioral Cloning
  8. Results
  9. Experiment Setup
  10. Design Task Training Dynamics and Circuit Optimization
  11. Ablations
  12. Comparisons with Baselines
  13. Computation Cost
  14. Conclusion
  15. Appendix
  16. ...and 7 more sections

Figures (9)

  • Figure 1: Block diagram of the AstRL design framework.
  • Figure 2: Overview of the graph representation of a circuit topology. (1) Raw circuit schematic for a differential resistive-loaded amplifier; (2) Diagram of a transistor with pins detailed; (3) Edge encoding with definition and translated feature; (4) Node encoding with definition and translated feature; (5) Final graph representation.
  • Figure 3: Overview of a single action, with masking. (1) Initial circuit; (2) Netlist converted to graph; (3) Source node selection; (4) Target node selection and addition; (5) Edge terminal selection and addition; (6) Action modification; (7) Termination decision; (8) Modified circuit.
  • Figure 4: Design task outputs. Figs. (a, b, c) show training dynamics and the number of generated designs. Figs. (d, e, f) show the shift of select performance specifications over training iterations for each task using fitted distributions.
  • Figure 5: Breakdown of dataset components.
  • ...and 4 more figures