Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Neural Control Barrier Functions from Expert Demonstrations using Inverse Constraint Learning

Yuxuan Yang, Hussein Sibai

TL;DR

The paper tackles safety for autonomous systems by learning neural Control Barrier Functions (CBFs) from expert demonstrations through inverse constraint learning (ICL). It introduces MT-ICL-based labeling of trajectories using a learned constraint $\hat{c}_$, and trains a neural CBF $B_ heta$ with a CBF-QP safety policy, iterating between labeling and learning with a practical heuristic to speed training. Across four dynamical systems (single integrator, inverted pendulum, Dubins car, quadrotor), the proposed ICL-CBF method outperforms baselines and approaches the performance of ground-truth safety labels, while offering scalable, real-time safety filtering. The approach provides a principled, data-driven path to safety guarantees in complex domains by leveraging expert demonstrations and inverse constraint inference.

Abstract

Safety is a fundamental requirement for autonomous systems operating in critical domains. Control barrier functions (CBFs) have been used to design safety filters that minimally alter nominal controls for such systems to maintain their safety. Learning neural CBFs has been proposed as a data-driven alternative for their computationally expensive optimization-based synthesis. However, it is often the case that the failure set of states that should be avoided is non-obvious or hard to specify formally, e.g., tailgating in autonomous driving, while a set of expert demonstrations that achieve the task and avoid the failure set is easier to generate. We use ICL to train a constraint function that classifies the states of the system under consideration to safe, i.e., belong to a controlled forward invariant set that is disjoint from the unspecified failure set, and unsafe ones, i.e., belong to the complement of that set. We then use that function to label a new set of simulated trajectories to train our neural CBF. We empirically evaluate our approach in four different environments, demonstrating that it outperforms existing baselines and achieves comparable performance to a neural CBF trained with the same data but annotated with ground-truth safety labels.

Learning Neural Control Barrier Functions from Expert Demonstrations using Inverse Constraint Learning

TL;DR

The paper tackles safety for autonomous systems by learning neural Control Barrier Functions (CBFs) from expert demonstrations through inverse constraint learning (ICL). It introduces MT-ICL-based labeling of trajectories using a learned constraint , and trains a neural CBF with a CBF-QP safety policy, iterating between labeling and learning with a practical heuristic to speed training. Across four dynamical systems (single integrator, inverted pendulum, Dubins car, quadrotor), the proposed ICL-CBF method outperforms baselines and approaches the performance of ground-truth safety labels, while offering scalable, real-time safety filtering. The approach provides a principled, data-driven path to safety guarantees in complex domains by leveraging expert demonstrations and inverse constraint inference.

Abstract

Safety is a fundamental requirement for autonomous systems operating in critical domains. Control barrier functions (CBFs) have been used to design safety filters that minimally alter nominal controls for such systems to maintain their safety. Learning neural CBFs has been proposed as a data-driven alternative for their computationally expensive optimization-based synthesis. However, it is often the case that the failure set of states that should be avoided is non-obvious or hard to specify formally, e.g., tailgating in autonomous driving, while a set of expert demonstrations that achieve the task and avoid the failure set is easier to generate. We use ICL to train a constraint function that classifies the states of the system under consideration to safe, i.e., belong to a controlled forward invariant set that is disjoint from the unspecified failure set, and unsafe ones, i.e., belong to the complement of that set. We then use that function to label a new set of simulated trajectories to train our neural CBF. We empirically evaluate our approach in four different environments, demonstrating that it outperforms existing baselines and achieves comparable performance to a neural CBF trained with the same data but annotated with ground-truth safety labels.
Paper Structure (23 sections, 1 theorem, 11 equations, 5 figures, 3 tables, 1 algorithm)

This paper contains 23 sections, 1 theorem, 11 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Learning CBFs from expert demonstrations
  4. Inverse constraint learning
  5. Preliminaries
  6. Control Barrier Functions
  7. Inverse constraint learning
  8. Training neural CBFs using ICL
  9. Proposed algorithm
  10. Heuristic for approximating $\hat{\pi}_E$ during training
  11. Experimental results
  12. Setup
  13. Implementation Details
  14. Single integrator
  15. Inverted pendulum
  16. ...and 8 more sections

Key Result

Theorem 1

Fix a CBF $B$ for system (eq:system) where $\nabla B(x) \neq 0$ for all $x\in B_{= 0} := \{x\ |\ B(x) = 0\}$. Then, any Lipshitz continuous control policy $\pi:\mathcal{X}\rightarrow \mathcal{U}$ that satisfies will result in $B_{\geq 0}$ being a forward invariant set, i.e., any trajectory of (eq:system) starting in $B_{\geq 0}$ remains inside it $\forall t \geq 0$.

Figures (5)

  • Figure 1: The framework for training an ICL-CBF $B_\theta$, where the solid arrows represent the expert trajectories and the dashed ones represent the ones generated using a CBF-QP policy consisting of $\pi_{\mathit{ref}}$ and $B_\theta$. First, we train $B_\theta$ using trajectory data sampled with $\pi_\mathit{ref}$ that are labeled using the learned constraint function $\hat{c}_\phi$, and simulate the system while using it to filter unsafe actions. Then, we compute the difference between the sampled trajectories and their corresponding expert ones and update $\hat{c}_\phi$ accordingly. Third, we retrain $B_\theta$ using a new set of sampled trajectories that are labeled using the new $\hat{c}_\phi$, simulate again, return to the second step, and repeat until convergence.
  • Figure 2: Values of the learned constraint and neural CBFs in the single-integrator scenario, where the black circles in the middle of the figures represent the obstacle. For (a), we color the boundary of the set $\{\boldsymbol{\mathrm{x}}\in\mathbb{R}^2|\hat{c}_\phi(\boldsymbol{\mathrm{x}})=\delta\}$ in red and the blue points are randomly sampled states from the expert demonstrations. For (b)-(f), the red margins denote the zero level set of each (neural) CBF.
  • Figure 3: The neural CBFs learned with different methods along with trajectories of the inverted pendulum starting from sampled initial states and following corresponding CBF-QP policies. The red points are initial states, the green points (at the origin) are the goal states, and the red contours represent the zero level set of each neural CBF.
  • Figure 4: Training data generated with different methods for inverted pendulum, where states within the black margins belong to the predefined safe set. Red points represent unsafe states, while blue points indicate safe states as annotated by each method.
  • Figure 5: Collision and success rates as $\delta$ varies in the Single integrator and Dubins car scenarios.

Theorems & Definitions (3)

  • Definition 1: Zeroing control barrier functions cbf
  • Theorem 1: cbf
  • Remark 1