Lyapunov Neural Network with Region of Attraction Search

TL;DR

A specific choice of Lyapunov function architecture that ensures non-negativity and a unique global minimum by construction is proposed and can be leveraged to find the controller and Lyapunov certificates faster and with a larger valid region by maximizing the size of a square inscribed in a given level set.

Abstract

Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive empirical performance, they lack the stability guarantees that are important in safety-critical situations. One way to provide these guarantees is to learn Lyapunov certificates alongside control policies. There are three related problems: 1) verify that a given Lyapunov function candidate satisfies the conditions for a given controller on a region, 2) find a valid Lyapunov function and controller on a given region, and 3) find a valid Lyapunov function and a controller such that the region of attraction is as large as possible. Previous work has shown that if the dynamics are piecewise linear, it is possible to solve problems 1) and 2) by solving a Mixed-Integer Linear Program (MILP). In this work, we build upon this method by proposing a Lyapunov neural network that considers monotonicity over half spaces in different directions. We 1) propose a specific choice of Lyapunov function architecture that ensures non-negativity and a unique global minimum by construction, and 2) show that this can be leveraged to find the controller and Lyapunov certificates faster and with a larger valid region by maximizing the size of a square inscribed in a given level set. We apply our method to a 2D inverted pendulum, unicycle path following, a 3-D feedback system, and a 4-D cart pole system, and demonstrate it can shorten the training time by half compared to the baseline, as well as find a larger ROA.

Figures (8)

• Figure 1: Illustration of Monotonic Layer $V(x)=[M]_1=\sum_{i=1}^{3}m_i(v_i^T x)$ with $v_i\in\{[1;0],[-1;1],[-1;-1]\}$, each $m_i(\cdot)$ is shown in (b).
• Figure 2: Visualization for the proof of proposition \ref{['proposition:bounded_domain_fromSet']}.
• Figure 3: Illustration of level set expansion on 1D Lyapunov functions: the blue box (identified with $l^\star$) is the original level set and the red one (identified with $\hat{l}^\star$) is after expansion.
• Figure 4: Final Lyapunov function and example controlled trajectories after applying Algo. \ref{['algo:verfication_in_set']}. (bold red curve) ROA within the bounded domain. (red dot) Equilibrium point. (green dots) Randomly selected initial conditions, and the corresponding system trajectories under the learned controller.
• Figure 5: Illustration of ROA in the inverted pendulum example. (a) $\mathcal{R}_{3.0}$ before expansion is highlighted in blue. (b) $\mathcal{R}_{3.0}$ after expansion is in red; the $l_{\infty}$ norm is shown as the inscribed blue box, and the intersection between $l_{\infty}$ norm and $\mathcal{R}_{3.0}$ is shown as a red dot. The level set touching the state space bounds is shown in green ($\mathcal{R}_{2.3}$). (c) Trajectories. (d) Lyapunov values.

Theorems & Definitions (17)

• Definition 1: Control Lyapunov Functions
• Remark 1
• Definition 2: Monotone functions
• Definition 3: Monotonic unit
• Proposition 1
• proof
• Definition 4: Monotonic Layer
• Lemma 1
• Definition 5: Lyapunov Neural Network
• Proposition 2