Learning to Stabilize High-dimensional Unknown Systems Using Lyapunov-guided Exploration

TL;DR

The paper tackles stabilizing high-dimensional unknown dynamical systems where full dynamics are unavailable or impractical to model. It introduces LYGE, a framework that learns local dynamics, a neural Control Lyapunov Function, and a stabilizing controller while Lyapunov-guided exploration selectively collects data within a growing trusted region toward the goal, ensuring stability within convergence. Empirically, LYGE achieves comparable or better stabilization than RL/IL baselines across six environments, notably reducing sample requirements by roughly 68%–95%, and scales to complex 16D/4D F-16 models. The approach also extends to learning alternative certificates, such as Control Contraction Metrics, highlighting its generality for stabilizing unknown systems where dynamics are partially learned. Overall, LYGE offers a practical, scalable route to data-efficient stabilization without requiring a full dynamical model, with promising avenues for formal verification and certificate-based extensions.

Abstract

Designing stabilizing controllers is a fundamental challenge in autonomous systems, particularly for high-dimensional, nonlinear systems that can hardly be accurately modeled with differential equations. The Lyapunov theory offers a solution for stabilizing control systems, still, current methods relying on Lyapunov functions require access to complete dynamics or samples of system executions throughout the entire state space. Consequently, they are impractical for high-dimensional systems. This paper introduces a novel framework, LYapunov-Guided Exploration (LYGE), for learning stabilizing controllers tailored to high-dimensional, unknown systems. LYGE employs Lyapunov theory to iteratively guide the search for samples during exploration while simultaneously learning the local system dynamics, control policy, and Lyapunov functions. We demonstrate its scalability on highly complex systems, including a high-fidelity F-16 jet model featuring a 16D state space and a 4D input space. Experiments indicate that, compared to prior works in reinforcement learning, imitation learning, and neural certificates, LYGE reduces the distance to the goal by 50% while requiring only 5% to 32% of the samples. Furthermore, we demonstrate that our algorithm can be extended to learn controllers guided by other certificate functions for unknown systems.

Figures (7)

• Figure 1: Trajectories generated by LYGE in different iterations in inverted pendulum environment. The counters show the learned CLF. The orange trajectories are generated by the learned controller in the current iteration. The light orange dots are the demonstrations generated in previous iterations, which also indicate the trusted tunnel $\mathcal{H}$. The black dot is the goal.
• Figure 2: The distance to the goal w.r.t. time step of LYGE and the baselines: (a) Inv Pendulum; (b) Cart Pole; (c) Cart II Pole; (d) Neural Lander; (e) F-16 GCA; (f) F-16 Tracking. The solid lines show the mean distance while the shaded regions show the standard deviation. Note that the curve corresponding to CLF-sparse almost overlaps with the curve corresponding to CLF-dense and so, the curve for CLF-sparse might not be visible in some of the plots.
• Figure 3: Ablation studies. (a) The converged reward w.r.t. demonstration rewards. (b) The number of samples used before LYGE converges w.r.t. demonstration rewards. (c) The converged reward w.r.t. $\epsilon$. (d) The converged reward w.r.t. $\lambda$. (e) The converged reward w.r.t. $\eta_\mathrm{ctrl}$.
• Figure 4: The tracking error w.r.t. time step in Dubins car path tracking environment.
• Figure 5: The expected return of LYGE and the baselines with respect to the number of samples. The dashed red line shows the converged reward of LYGE. In the right of each subplot, we show the whole curve of the expected return w.r.t. the number of samples, and since our LYGE converges too fast compared with the baselines, we zoom in the region inside the dashed rectangle and show this region in the left. The triangle on the x-axis shows the number of samples needed by LYGE to converge. The reward at $0$ number of samples is the reward of demonstrations.

Theorems & Definitions (15)

• definition 1
• proposition 1
• proof
• definition 2
• definition 3
• lemma 1
• proof
• lemma 2
• proof
• theorem 1