Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation

Luca Marzari, Francesco Trotti, Enrico Marchesini, Alessandro Farinelli

TL;DR

The paper tackles safe mapless navigation by marrying offline probabilistic enumeration of unsafe regions with a control barrier function (CBF) safety layer that corrects DRL policy actions via a quadratic program and then tracks safe references through nonlinear model predictive control. The approach yields forward-invariant safe sets independent of the deployment environment, enabling zero safety violations while preserving navigation efficiency in both simulation and real-world tests, including an aquatic drone and a Turtlebot3. Key contributions include (i) probabilistic safety identification to form a global safe set, (ii) a CBF-based corrective layer that minimally adjusts policy actions, and (iii) an NMPC-based low-level controller ensuring robust, safe, and efficient navigation under uncertainty. The work demonstrates practical impact by enabling safer deployment of DRL policies in complex, uncertain real-world settings without extensive domain-specific tailoring.

Abstract

Achieving safe autonomous navigation systems is critical for deploying robots in dynamic and uncertain real-world environments. In this paper, we propose a hierarchical control framework leveraging neural network verification techniques to design control barrier functions (CBFs) and policy correction mechanisms that ensure safe reinforcement learning navigation policies. Our approach relies on probabilistic enumeration to identify unsafe regions of operation, which are then used to construct a safe CBF-based control layer applicable to arbitrary policies. We validate our framework both in simulation and on a real robot, using a standard mobile robot benchmark and a highly dynamic aquatic environmental monitoring task. These experiments demonstrate the ability of the proposed solution to correct unsafe actions while preserving efficient navigation behavior. Our results show the promise of developing hierarchical verification-based systems to enable safe and robust navigation behaviors in complex scenarios.

Designing Control Barrier Function via Probabilistic Enumeration for Safe Reinforcement Learning Navigation

TL;DR

The paper tackles safe mapless navigation by marrying offline probabilistic enumeration of unsafe regions with a control barrier function (CBF) safety layer that corrects DRL policy actions via a quadratic program and then tracks safe references through nonlinear model predictive control. The approach yields forward-invariant safe sets independent of the deployment environment, enabling zero safety violations while preserving navigation efficiency in both simulation and real-world tests, including an aquatic drone and a Turtlebot3. Key contributions include (i) probabilistic safety identification to form a global safe set, (ii) a CBF-based corrective layer that minimally adjusts policy actions, and (iii) an NMPC-based low-level controller ensuring robust, safe, and efficient navigation under uncertainty. The work demonstrates practical impact by enabling safer deployment of DRL policies in complex, uncertain real-world settings without extensive domain-specific tailoring.

Abstract

Achieving safe autonomous navigation systems is critical for deploying robots in dynamic and uncertain real-world environments. In this paper, we propose a hierarchical control framework leveraging neural network verification techniques to design control barrier functions (CBFs) and policy correction mechanisms that ensure safe reinforcement learning navigation policies. Our approach relies on probabilistic enumeration to identify unsafe regions of operation, which are then used to construct a safe CBF-based control layer applicable to arbitrary policies. We validate our framework both in simulation and on a real robot, using a standard mobile robot benchmark and a highly dynamic aquatic environmental monitoring task. These experiments demonstrate the ability of the proposed solution to correct unsafe actions while preserving efficient navigation behavior. Our results show the promise of developing hierarchical verification-based systems to enable safe and robust navigation behaviors in complex scenarios.
Paper Structure (18 sections, 1 theorem, 12 equations, 8 figures)

This paper contains 18 sections, 1 theorem, 12 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. Reinforcement Learning and Constrained Approaches
  4. Formal Verification of Neural Networks
  5. Control Barrier Function
  6. Dynamic Models
  7. Mobile robot
  8. Aquatic drone
  9. Problem Statement
  10. Methodology
  11. Compute the Forward Invariant Safe Set
  12. Control Barrier Function and Low-Level Controller
  13. Control Barrier Function
  14. Low-level controller
  15. Empirical Evaluation
  16. ...and 3 more sections

Key Result

Proposition 1

The set $\mathcal{C} = \bigcup_i \mathcal{C}_i$ is a valid safe set for a control barrier function.

Figures (8)

  • Figure 1: Overview of the proposed approach.
  • Figure 2: Illustrative example of a safety property in aquatic mapless navigation: the left shows an unsafe state near the coastline, while the right defines a region of the state space $\hat{\mathcal{S}} \subset \mathcal{S}$ to verify the agents never choose a leftward movement.
  • Figure 3: Block diagram of the hierarchical architecture.
  • Figure 4: Explanatory example of the relationship between the enumeration process and the CBF. On the left, we report an unsafe situation where the agent has two obstacles on the left and the coastline on the right. In the center, we define the input region to be verified in green, and we enumerate the unsafe regions where the agent has a high probability of colliding. Finally, on the right, we report the translation of the enumeration result into the CBF's formulation.
  • Figure 5: Linear and angular velocities tracking via NMPC. Red dashed lines are the reference $r$ while the blue lines are the optimal control action of the NMPC $u^{*}$.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Definition 1: Control Barrier Function cbf
  • Proposition 1