Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ensuring Safety in Target Pursuit Control: A CBF-Safe Reinforcement Learning Approach

Yaosheng Deng, Junjie Gao, Jiaping Xiao, Mir Feroskhan

TL;DR

This work addresses safe target-pursuit in multi-agent settings by enforcing collision avoidance, sensing range, and input constraints. It introduces CSRL, a framework that couples a trained RL policy with a safety filter formed by three adaptive CBFs and a switch strategy to guarantee safety under disturbances. The safety filter solves a QP to convert unsafe RL actions into safe controls, and a feasibility theorem with KKT conditions ensures safe operation for all pursuers. Simulations with UAV-style agents show CSRL maintains safety while achieving robust tracking in complex, disturbance-rich scenarios, outperforming RL-only approaches. The approach offers a practical, scalable path to safe, real-time pursuit in dynamic environments.

Abstract

This paper addresses the target-pursuit problem, aiming to ensure each pursuer's safety regarding collision avoidance, sensing range, and input saturation. An input-constrained CBF is proposed to dynamically regulate the pursuer's control, ensuring effective target pursuit even when the target performs evasive maneuvers. To further ensure safety, two sets of CBF constraints are designed to regulate the pursuer's position, enabling it to keep the target within the sensing range while avoiding collision in complex environments with external disturbances. These three CBFs collectively form our safety filter, which filters unsafe outputs from RL by solving a Quadratic Program (QP). Finally, the safety filter, combined with a switch strategy that enhances the feasibility of solving its QP, constitutes the Control Barrier Function (CBF)-Safe Reinforcement Learning (CSRL) algorithm, whose solutions are proven to satisfy the Karush-Kuhn-Tucker (KKT) conditions for all safety constraints. Simulation results validate the effectiveness of the CSRL algorithm, demonstrating its ability to handle complex pursuit scenarios while maintaining safety and improving control performance.

Ensuring Safety in Target Pursuit Control: A CBF-Safe Reinforcement Learning Approach

TL;DR

This work addresses safe target-pursuit in multi-agent settings by enforcing collision avoidance, sensing range, and input constraints. It introduces CSRL, a framework that couples a trained RL policy with a safety filter formed by three adaptive CBFs and a switch strategy to guarantee safety under disturbances. The safety filter solves a QP to convert unsafe RL actions into safe controls, and a feasibility theorem with KKT conditions ensures safe operation for all pursuers. Simulations with UAV-style agents show CSRL maintains safety while achieving robust tracking in complex, disturbance-rich scenarios, outperforming RL-only approaches. The approach offers a practical, scalable path to safe, real-time pursuit in dynamic environments.

Abstract

This paper addresses the target-pursuit problem, aiming to ensure each pursuer's safety regarding collision avoidance, sensing range, and input saturation. An input-constrained CBF is proposed to dynamically regulate the pursuer's control, ensuring effective target pursuit even when the target performs evasive maneuvers. To further ensure safety, two sets of CBF constraints are designed to regulate the pursuer's position, enabling it to keep the target within the sensing range while avoiding collision in complex environments with external disturbances. These three CBFs collectively form our safety filter, which filters unsafe outputs from RL by solving a Quadratic Program (QP). Finally, the safety filter, combined with a switch strategy that enhances the feasibility of solving its QP, constitutes the Control Barrier Function (CBF)-Safe Reinforcement Learning (CSRL) algorithm, whose solutions are proven to satisfy the Karush-Kuhn-Tucker (KKT) conditions for all safety constraints. Simulation results validate the effectiveness of the CSRL algorithm, demonstrating its ability to handle complex pursuit scenarios while maintaining safety and improving control performance.

Paper Structure

This paper contains 13 sections, 5 theorems, 69 equations, 21 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. CBF
  4. High-order CBF (HOCBF)
  5. Problem Statement
  6. Safe Strategy for Pursuers
  7. Statement of Problem
  8. Methodology
  9. System transform
  10. Safety Filter Design
  11. CSRL Algorithm
  12. Experiments
  13. Conclusion

Key Result

Lemma 1

Let $t_j\in[t-\Delta T,t]\subset\mathbb{R}$, $j\in\{1,\ldots,M\}$ be the $j$-th sampling time. $M$ represents the total sampling times. Given the adaptive law for system sys-aug as follows where, $\Delta x_{i,j}(t) = x_i(t_j)-x_i(t_j-\Delta T)$, $\mathcal{F}_i(t_j)=\int_{t_j-\Delta T}^{t_j}f(x_i(\tau))d\tau$, $\mathcal{Y}_i(t_j)=\int_{t_j-\Delta T}^{t_j}Y(x_i(\tau))d\tau$, $\mathcal{G}_i(t_j) = \

Figures (21)

  • Figure 1: The average return training curves of SAC by running $3$ times with different seeds. The lines and shaded areas represent the average return and the 95% confidence interval, respectively.
  • Figure 2: CSRL at $t=73$s.
  • Figure 3: CSRL at $t=77$s.
  • Figure 4: CSRL at $t=600$s.
  • Figure 6: SAC at $t=73$s
  • ...and 16 more figures

Theorems & Definitions (15)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Remark 1
  • Lemma 1
  • Remark 2
  • Remark 3
  • Lemma 2
  • Lemma 3
  • ...and 5 more