Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistically Assuring Safety of Control Systems using Ensembles of Safety Filters and Conformal Prediction

Ihab Tabbara, Yuxuan Yang, Hussein Sibai

TL;DR

This work uses CP to calibrate the switching between the unsafe nominal controller and the learned HJ-based safe policy and to derive safety guarantees under this switched policy, and investigates using an ensemble of independently trained HJ value functions as a safety filter.

Abstract

Safety assurance is a fundamental requirement for deploying learning-enabled autonomous systems. Hamilton-Jacobi (HJ) reachability analysis is a fundamental method for formally verifying safety and generating safe controllers. However, computing the HJ value function that characterizes the backward reachable set (BRS) of a set of user-defined failure states is computationally expensive, especially for high-dimensional systems, motivating the use of reinforcement learning approaches to approximate the value function. Unfortunately, a learned value function and its corresponding safe policy are not guaranteed to be correct. The learned value function evaluated at a given state may not be equal to the actual safety return achieved by following the learned safe policy. To address this challenge, we introduce a conformal prediction-based (CP) framework that bounds such uncertainty. We leverage CP to provide probabilistic safety guarantees when using learned HJ value functions and policies to prevent control systems from reaching failure states. Specifically, we use CP to calibrate the switching between the unsafe nominal controller and the learned HJ-based safe policy and to derive safety guarantees under this switched policy. We also investigate using an ensemble of independently trained HJ value functions as a safety filter and compare this ensemble approach to using individual value functions alone.

Statistically Assuring Safety of Control Systems using Ensembles of Safety Filters and Conformal Prediction

TL;DR

This work uses CP to calibrate the switching between the unsafe nominal controller and the learned HJ-based safe policy and to derive safety guarantees under this switched policy, and investigates using an ensemble of independently trained HJ value functions as a safety filter.

Abstract

Safety assurance is a fundamental requirement for deploying learning-enabled autonomous systems. Hamilton-Jacobi (HJ) reachability analysis is a fundamental method for formally verifying safety and generating safe controllers. However, computing the HJ value function that characterizes the backward reachable set (BRS) of a set of user-defined failure states is computationally expensive, especially for high-dimensional systems, motivating the use of reinforcement learning approaches to approximate the value function. Unfortunately, a learned value function and its corresponding safe policy are not guaranteed to be correct. The learned value function evaluated at a given state may not be equal to the actual safety return achieved by following the learned safe policy. To address this challenge, we introduce a conformal prediction-based (CP) framework that bounds such uncertainty. We leverage CP to provide probabilistic safety guarantees when using learned HJ value functions and policies to prevent control systems from reaching failure states. Specifically, we use CP to calibrate the switching between the unsafe nominal controller and the learned HJ-based safe policy and to derive safety guarantees under this switched policy. We also investigate using an ensemble of independently trained HJ value functions as a safety filter and compare this ensemble approach to using individual value functions alone.

Paper Structure

This paper contains 23 sections, 2 theorems, 12 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Hamilton–Jacobi reachability analysis
  4. Conformal prediction
  5. Problem Setup
  6. Methodology
  7. Phase 1 (conformal prediction-based switching)
  8. Calibration dataset construction
  9. One-sided calibration
  10. Switched policy
  11. Ensemble of HJ value functions
  12. Phase 2 (safety verification using conformal prediction)
  13. Case Study
  14. Task description
  15. Analysis of single-model safety filters
  16. ...and 8 more sections

Key Result

Lemma 1

Given a user-defined miscoverage rate $\alpha \in (0,1)$, for any $t\in \mathbb{N}$, the prediction interval $\mathcal{C}(x_{t+1}, \alpha) := [V_{\theta}(x_{t+1}) - \hat{q}(\alpha), \infty)$ contains $V_{\theta}^*(x_{t+1})$ with probability at least $1 - \alpha$: This is a marginal guarantee: the probability is taken over the joint draw of the calibration set and the test point, conditional on th

Figures (3)

  • Figure 1: Violation rates (blue bars) and success rates (orange bars) achieved in 50 trials using an ensemble of HJ value functions as safety filters following the multiple strategy in Algorithm \ref{['alg:ensemble_safety_filter_strategies']} and its corresponding member models following Algorithm \ref{['alg:single_model_safety_filter']} for different choices of coverage rates $\alpha$.
  • Figure 2: (a) The success rate of the safety filters using an ensemble of HJ value functions with different switching strategies (single and multiple) and with different choices of $\alpha$. The dots in the plot represent the success rates (left figure) and the violation rates (right figure) for the different values of $\alpha$ shown at the perimeter of the circle. The center dot denotes 0, and each black circle represents an increase in value, with a step size of 0.2 for Success and 0.04 for Violation. (b) Graph of $\mathbb{P}_{x_0 \sim \mathcal{P}_0}(J^{\pi_\text{ensemble}^{sw}}(x_0) > 0) \sim \text{Beta}(N_{\text{cert}} - k, k + 1)$. For each curve, we ran $N_{cert}$ simulations and calculated $k$, the number of simulations where the agent enters the failure set at some time instant in the trajectory, and then plotted the corresponding Beta distribution.
  • Figure 3: Triple-vehicle Highway Takeover Environment

Theorems & Definitions (4)

  • Definition 1
  • Lemma 1
  • theorem 1
  • proof