RNBF: Real-Time RGB-D Based Neural Barrier Functions for Safe Robotic Navigation

TL;DR

This work tackles safe navigation in unknown environments using only low-cost RGB-D sensing by online learning a continuous neural SDF that serves as a barrier function for CBF-based controllers. The RNBF pipeline adds floor segmentation and a huber-based near-surface loss to online SDF reconstruction, producing differentiable barriers at 5–15 Hz that feed a CBF-QP safety filter running at higher frequency. Compared to baselines, RNBF achieves collision-free, goal-reaching behavior in both simulated and real Fetch robot experiments, while robustly handling depth noise and floor geometry. The approach preserves compatibility with established SDF-based reactive controllers and offers a practical safety layer for autonomous navigation in novel environments, with modular design for broader robotic applications.

Abstract

Autonomous safe navigation in unstructured and novel environments poses significant challenges, especially when environment information can only be provided through low-cost vision sensors. Although safe reactive approaches have been proposed to ensure robot safety in complex environments, many base their theory off the assumption that the robot has prior knowledge on obstacle locations and geometries. In this paper, we present a real-time, vision-based framework that constructs continuous, first-order differentiable Signed Distance Fields (SDFs) of unknown environments entirely online, without any pre-training, and is fully compatible with established SDF-based reactive controllers. To achieve robust performance under practical sensing conditions, our approach explicitly accounts for noise in affordable RGB-D cameras, refining the neural SDF representation online for smoother geometry and stable gradient estimates. We validate the proposed method in simulation and real-world experiments using a Fetch robot. Videos and supplementary material are available at https://satyajeetburla.github.io/rnbf/.

Figures (9)

• Figure 1: Hardware setup for real-world experiments showing the RNBF-Control pipeline navigating around static and quasi-static obstacles without any prior knowledge about the obstacles.
• Figure 2: Block diagram of the proposed RNBF-based safe reactive navigation pipeline. The SDF Generation module (5–15 Hz) constructs a continuous, differentiable signed distance field from real-time RGB-D input, while the Reactive Controller ($\geq 20$ Hz) ensures collision-free navigation using a CBF-QP. The black thick arrow highlights the main coupling step between the perception and control pipelines. Red outlines highlight components in the SDF generation stage that distinguish the RNBF pipeline from the original iSDF pipeline.
• Figure 3: Architecture of the neural SDF network $h_\text{sdf}(\xi)$. The model is a fully connected MLP composed of four hidden layers, each with 256 units (depicted as blue rectangles). Yellow elliptical blocks marked “PE” represent positional encodings, where input coordinates $\xi \in \mathbb{R}^3$ are transformed via a periodic embedding $\gamma(\xi) \in \mathbb{R}^{297}$ to enable high-frequency detail reconstruction. The final linear layer outputs the scalar signed distance value $h_\text{sdf}(\xi) \in \mathbb{R}$.
• Figure 4: (a) Simulation environment setup with four different scenarios. (b) Ground-truth SDF used by Parametric CBF-QP. Note, RNBF-CBF-QP builds its SDF online and does not use the ground-truth SDF.
• Figure 5: Modified point of interest on Fetch robot.