How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints
Kensuke Nakamura, Arun L. Bishop, Steven Man, Aaron M. Johnson, Zachary Manchester, Andrea Bajcsy
TL;DR
This work tackles the challenge of safe visuomotor control when models are hard to learn and perception is high-dimensional. It reveals two key gaps: non-smooth margins from classifier-based failure encodings and a distribution mismatch between safety training data and deployment actions, which hinder smooth, optimization-based safety filtering. To address this, the authors introduce LatentCBF, combining gradient-penalized, Lipschitz-constrained margin learning inspired by Wasserstein disentanglement with a mixed-policy RL training regimen, enabling a latent, discrete-time CBF that can modulate a nominal policy in real time. Empirical results in both simulation and hardware demonstrate that LatentCBF yields smoother safety interventions and substantially improves safe-task success (e.g., 80% vs 38% in hardware) while maintaining high safety rates, and can scale to thousands of candidate actions with ~10 ms latency, offering practical safety for vision-based manipulation without hand-crafted models or full state observability.
Abstract
Latent safety filters extend Hamilton-Jacobi (HJ) reachability to operate on latent state representations and dynamics learned directly from high-dimensional observations, enabling safe visuomotor control under hard-to-model constraints. However, existing methods implement "least-restrictive" filtering that discretely switch between nominal and safety policies, potentially undermining the task performance that makes modern visuomotor policies valuable. While reachability value functions can, in principle, be adapted to be control barrier functions (CBFs) for smooth optimization-based filtering, we theoretically and empirically show that current latent-space learning methods produce fundamentally incompatible value functions. We identify two sources of incompatibility: First, in HJ reachability, failures are encoded via a "margin function" in latent space, whose sign indicates whether or not a latent is in the constraint set. However, representing the margin function as a classifier yields saturated value functions that exhibit discontinuous jumps. We prove that the value function's Lipschitz constant scales linearly with the margin function's Lipschitz constant, revealing that smooth CBFs require smooth margins. Second, reinforcement learning (RL) approximations trained solely on safety policy data yield inaccurate value estimates for nominal policy actions, precisely where CBF filtering needs them. We propose the LatentCBF, which addresses both challenges through gradient penalties that lead to smooth margin functions without additional labeling, and a value-training procedure that mixes data from both nominal and safety policy distributions. Experiments on simulated benchmarks and hardware with a vision-based manipulation policy demonstrate that LatentCBF enables smooth safety filtering while doubling the task-completion rate over prior switching methods.