Providing Safety Assurances for Systems with Unknown Dynamics

TL;DR

This work tackles safety guarantees for autonomous systems with dynamics learned from data. It proposes a framework that combines ensemble neural networks to model nominal dynamics with state- and control-dependent uncertainty bounds, and then solves a robust Hamilton-Jacobi reachability problem to compute the maximal robust safe set $\mathcal{R}^*$ and a corresponding controller $\pi_{\mathcal{R}^*}$. By explicitly accounting for how model uncertainty interacts with control, the method yields less conservative yet robust safety assurances and is instantiated with neural ensembles. Experiments on an inverted pendulum and a TurtleBot demonstrate improved safety robustness over baselines and illustrate practical viability for learned-dynamics safety filtering.

Abstract

As autonomous systems become more complex and integral in our society, the need to accurately model and safely control these systems has increased significantly. In the past decade, there has been tremendous success in using deep learning techniques to model and control systems that are difficult to model using first principles. However, providing safety assurances for such systems remains difficult, partially due to the uncertainty in the learned model. In this work, we aim to provide safety assurances for systems whose dynamics are not readily derived from first principles and, hence, are more advantageous to be learned using deep learning techniques. Given the system of interest and safety constraints, we learn an ensemble model of the system dynamics from data. Leveraging ensemble uncertainty as a measure of uncertainty in the learned dynamics model, we compute a maximal robust control invariant set, starting from which the system is guaranteed to satisfy the safety constraints under the condition that realized model uncertainties are contained in the predefined set of admissible model uncertainty. We demonstrate the effectiveness of our method using a simulated case study with an inverted pendulum and a hardware experiment with a TurtleBot. The experiments show that our method robustifies the control actions of the system against model uncertainty and generates safe behaviors without being overly restrictive. The codes and accompanying videos can be found on the project website.

Figures (4)

• Figure 1: Recovered safe sets (states starting from which the inverted pendulum stays within $0.6\pi$ of the upright for 0.7 seconds) for our method, the Mean Dynamics baseline (Baseline 1), the Partial Game baseline (Baseline 3) in the inverted pendulum experiment, with the ensemble trained with 300 training samples. Note that Baseline 2 (conformal prediction) is not visualized, because its safe set is empty.
• Figure 2: Changes in percent safe set recovered with our method, the Conformal Prediction baseline (Baseline 2), and the Partial Game baseline (Baseline 3), over the number of training samples.
• Figure 3: Comparison between projected safe sets $\mathcal{S}$ and $\mathcal{S}_m$ along with TurtleBot rollout trajectories using safety controllers $\pi^*$ and $\pi^*_m$. The initial state of rollouts using $\pi^*$ are marked with pentagons. All rollouts start with heading $\theta=0.7$ and angular velocity $\omega=0$. The states at which the TurtleBot enters failure set $\mathcal{F}$ are marked with crosses.
• Figure 4: Filtered controller $\hat{\pi}$ and $\hat{\pi}_d$ rollout trajectories. The blue markers indicate the states at which the nominal controller $\pi$ is on. Whereas the mint markers indicates the states where safety controllers $\pi^*$ and $\pi^*_{d}$ intervene. The state at which the vehicle exits the experiment space under $\hat{\pi}_d$ is marked with a red cross.

Theorems & Definitions (2)

• Definition 1
• Lemma 1