Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Actor-Critic Physics-informed Neural Lyapunov Control

Jiarui Wang, Mahyar Fazlyab

TL;DR

This work addresses stabilizing nonlinear systems with provable guarantees while maximizing the region of attraction (DoA) under actuation constraints. It introduces a physics-informed actor-critic framework that co-learns a neural policy $\\pi_\\gamma$ and a Zubov function $W_\\theta$, guided by Zubov's PDE to shape the DoA boundary and certify stability. A vertex-softmax parameterization enables constraint enforcement without projection layers, and formal verification via a solver certifies the learned DoA. Across benchmarks including the Double Integrator, Van der Pol, Inverted Pendulum, and Bicycle Tracking, the proposed method consistently enlarges the verifiable DoA compared to state-of-the-art approaches, highlighting its practical potential for safe, robust nonlinear control.

Abstract

Designing control policies for stabilization tasks with provable guarantees is a long-standing problem in nonlinear control. A crucial performance metric is the size of the resulting region of attraction, which essentially serves as a robustness "margin" of the closed-loop system against uncertainties. In this paper, we propose a new method to train a stabilizing neural network controller along with its corresponding Lyapunov certificate, aiming to maximize the resulting region of attraction while respecting the actuation constraints. Crucial to our approach is the use of Zubov's Partial Differential Equation (PDE), which precisely characterizes the true region of attraction of a given control policy. Our framework follows an actor-critic pattern where we alternate between improving the control policy (actor) and learning a Zubov function (critic). Finally, we compute the largest certifiable region of attraction by invoking an SMT solver after the training procedure. Our numerical experiments on several design problems show consistent and significant improvements in the size of the resulting region of attraction.

Actor-Critic Physics-informed Neural Lyapunov Control

TL;DR

This work addresses stabilizing nonlinear systems with provable guarantees while maximizing the region of attraction (DoA) under actuation constraints. It introduces a physics-informed actor-critic framework that co-learns a neural policy and a Zubov function , guided by Zubov's PDE to shape the DoA boundary and certify stability. A vertex-softmax parameterization enables constraint enforcement without projection layers, and formal verification via a solver certifies the learned DoA. Across benchmarks including the Double Integrator, Van der Pol, Inverted Pendulum, and Bicycle Tracking, the proposed method consistently enlarges the verifiable DoA compared to state-of-the-art approaches, highlighting its practical potential for safe, robust nonlinear control.

Abstract

Designing control policies for stabilization tasks with provable guarantees is a long-standing problem in nonlinear control. A crucial performance metric is the size of the resulting region of attraction, which essentially serves as a robustness "margin" of the closed-loop system against uncertainties. In this paper, we propose a new method to train a stabilizing neural network controller along with its corresponding Lyapunov certificate, aiming to maximize the resulting region of attraction while respecting the actuation constraints. Crucial to our approach is the use of Zubov's Partial Differential Equation (PDE), which precisely characterizes the true region of attraction of a given control policy. Our framework follows an actor-critic pattern where we alternate between improving the control policy (actor) and learning a Zubov function (critic). Finally, we compute the largest certifiable region of attraction by invoking an SMT solver after the training procedure. Our numerical experiments on several design problems show consistent and significant improvements in the size of the resulting region of attraction.
Paper Structure (22 sections, 3 theorems, 18 equations, 8 figures, 1 table, 1 algorithm)

This paper contains 22 sections, 3 theorems, 18 equations, 8 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Notation
  4. Background and Problem Statement
  5. Lyapunov Certificates
  6. Problem Statement
  7. Neural Lyapunov Control
  8. Physics-Informed Actor-Critic Learning
  9. Maximal Lyapunov Functions and Zubov's Method
  10. Learning the Zubov Function (Critic)
  11. Policy Improvement (Actor)
  12. Actor-Critic Learning
  13. Actuation Constraints
  14. Verification
  15. Numerical Experiments
  16. ...and 7 more sections

Key Result

Theorem 1

Consider the dynamical system in (1). If there exists a continuously differentiable function $V \colon \mathcal{D} \rightarrow \mathbb{R}$ satisfying the following conditions then the zero solution $x(t)=0$ is asymptotically stable.

Figures (8)

  • Figure 1: The level set $\{x : W(x)=1\}$ obtained by the Zubov PDE characterizes the boundary of the true DoA.
  • Figure 2: Double integrator system
  • Figure 3: Van der Pol system
  • Figure 4: Inverted Pendulum system
  • Figure 5: Bicycle tracking system
  • ...and 3 more figures

Theorems & Definitions (6)

  • Definition 1: StabilityHaddad_Chellaboina_2008
  • Definition 2: Asymptotic StabilityHaddad_Chellaboina_2008
  • Definition 3: Domain of Attraction Haddad_Chellaboina_2008
  • Theorem 1: Lyapunov StabilityHaddad_Chellaboina_2008
  • Theorem 2: Maximal Lyapunov Functionvannelli1985maximal
  • Theorem 3: Zubov's Theorem zubov1961methods