Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimization Algorithm Design via Electric Circuits

Stephen P. Boyd, Tetiana Parshakova, Ernest K. Ryu, Jaewook J. Suh

TL;DR

This work presents a framework that designs convex optimization algorithms by mapping continuous-time RLC circuit dynamics to optimization problems and then automatically discretizing them to obtain provably convergent discrete-time methods. The approach unifies circuit theory and optimization, using interconnects (static and dynamic) coupled to the subdifferential operator $oldsymbol{ abla f}$ to encode optimality conditions, and leverages the Performance Estimation Problem to guarantee dissipativity in discretizations. It recovers classical algorithms such as Nesterov acceleration, decentralized ADMM, and PG-EXTRA, while enabling the automated discovery of new methods (e.g., DADMM+C) with convergence proofs and empirical speedups. The methodology is supported by an open-source pipeline for automatic discretization and demonstrates robust performance on decentralized problems, suggesting broad applicability to distributed optimization and beyond.

Abstract

We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is to design an appropriate electric circuit whose continuous-time dynamics converge to the solution of the optimization problem at hand. Then, the second stage is an automated, computer-assisted discretization of the continuous-time dynamics, yielding a provably convergent discrete-time algorithm. Our methodology recovers many classical (distributed) optimization algorithms and enables users to quickly design and explore a wide range of new algorithms with convergence guarantees.

Optimization Algorithm Design via Electric Circuits

TL;DR

This work presents a framework that designs convex optimization algorithms by mapping continuous-time RLC circuit dynamics to optimization problems and then automatically discretizing them to obtain provably convergent discrete-time methods. The approach unifies circuit theory and optimization, using interconnects (static and dynamic) coupled to the subdifferential operator to encode optimality conditions, and leverages the Performance Estimation Problem to guarantee dissipativity in discretizations. It recovers classical algorithms such as Nesterov acceleration, decentralized ADMM, and PG-EXTRA, while enabling the automated discovery of new methods (e.g., DADMM+C) with convergence proofs and empirical speedups. The methodology is supported by an open-source pipeline for automatic discretization and demonstrates robust performance on decentralized problems, suggesting broad applicability to distributed optimization and beyond.

Abstract

We present a novel methodology for convex optimization algorithm design using ideas from electric RLC circuits. Given an optimization problem, the first stage of the methodology is to design an appropriate electric circuit whose continuous-time dynamics converge to the solution of the optimization problem at hand. Then, the second stage is an automated, computer-assisted discretization of the continuous-time dynamics, yielding a provably convergent discrete-time algorithm. Our methodology recovers many classical (distributed) optimization algorithms and enables users to quickly design and explore a wide range of new algorithms with convergence guarantees.

Paper Structure

This paper contains 80 sections, 13 theorems, 222 equations, 29 figures.

Table of Contents

  1. Introduction
  2. Optimization problem formulation.
  3. Example.
  4. Preliminaries and Notation
  5. Contributions
  6. Continuous-time optimization with circuits
  7. Interconnects
  8. Static interconnect.
  9. Dynamic interconnect.
  10. Admissibility.
  11. Composing interconnects with $\partial f$
  12. Equillibrium yields a primal-dual solution.
  13. Energy dissipation
  14. Circuits for classical algorithms
  15. Multi-wire notation.
  16. ...and 65 more sections

Key Result

theorem 1

Assume $f$ is $\mu$-strongly convex and $M$-smooth. Suppose $(v^0, i^0, x^0, y^0)$ satisfy Then there is a unique Lipschitz continuous curve $(v, i, x, y) \colon [0,\infty) \to \reals^\sigma \times \reals^\sigma \times \reals^m \times \reals^m$ satisfying the conditions in e-dyn-ic-partial-f and the initial condition $(v(0), i(0), x(0), y(0)) = (v^0, i^0, x^0, y^0)$.

Figures (29)

  • Figure 1: Example of a static interconnect, $m=5$, $N_1=\{1,3\}$, $N_2=\{2,4\}$, $N_3=\{5\}$.
  • Figure 2: Example of a dynamic interconnect with $\tau=8$ nodes, $\sigma=7$ RLC components, $m=5$ terminals, and $1$ ground node. Reduced node incidence matrix $A$ is provided. ($R_2$ and $R_3$ are $0$-ohm resistors.) This dynamic interconnect is admissible with respect to the static interconnect of Figure \ref{['fig-static-ic']}.
  • Figure 3: The static interconnect of Figure \ref{['fig-static-ic']} connected with $\partial f$. The potentials at the $m$ terminals is an optimal $x^\star\in \reals^m$ solving \ref{['e-dist-opt-primal']}.
  • Figure 4: The dynamic interconnect of Figure \ref{['fig-RLC-example']} connected with $\partial f$. The potentials at the $m$ terminals satisfy $x(t)\rightarrow x^\star$ for an optimal $x^\star\in \reals^m$ solving \ref{['e-dist-opt-primal']} under the conditions of Theorem \ref{['thm:convergence']}.
  • Figure 5: Multi-wire notation.
  • ...and 24 more figures

Theorems & Definitions (23)

  • theorem 1
  • theorem 2
  • lemma 1
  • theorem 3
  • lemma 2
  • proof
  • theorem 4
  • lemma 3
  • proof
  • corollary 1
  • ...and 13 more