Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CATNIPS: Collision Avoidance Through Neural Implicit Probabilistic Scenes

Timothy Chen, Preston Culbertson, Mac Schwager

TL;DR

This work tackles safe robot navigation in NeRF-based environments by recasting NeRFs as Poisson Point Processes, enabling rigorous computation of collision probabilities. It introduces PURR, a voxelized map that fuses NeRF density with robot geometry, and a two-stage planner (A$^*$-based pathfinding plus Bézier trajectory optimization) that guarantees safety with a user-specified collision probability. The approach, dubbed CATNIPS, delivers real-time trajectory planning (around $>3\,\mathrm{Hz}$ online) and outperforms prior NeRF-based planners in safety and conservatism, validated in both simulations and hardware. The PPP interpretation further offers a probabilistic perspective on NeRF training, uncertainty quantification, and active sensing, with broad implications for perception-guided planning and safe exploration in vision-based robotics.

Abstract

We introduce a transformation of a Neural Radiance Field (NeRF) to an equivalent Poisson Point Process (PPP). This PPP transformation allows for rigorous quantification of uncertainty in NeRFs, in particular, for computing collision probabilities for a robot navigating through a NeRF environment. The PPP is a generalization of a probabilistic occupancy grid to the continuous volume and is fundamental to the volumetric ray-tracing model underlying radiance fields. Building upon this PPP representation, we present a chance-constrained trajectory optimization method for safe robot navigation in NeRFs. Our method relies on a voxel representation called the Probabilistic Unsafe Robot Region (PURR) that spatially fuses the chance constraint with the NeRF model to facilitate fast trajectory optimization. We then combine a graph-based search with a spline-based trajectory optimization to yield robot trajectories through the NeRF that are guaranteed to satisfy a user-specific collision probability. We validate our chance constrained planning method through simulations and hardware experiments, showing superior performance compared to prior works on trajectory planning in NeRF environments. Our codebase can be found at https://github.com/chengine/catnips, and videos can be found on our project page (https://chengine.github.io/catnips).

CATNIPS: Collision Avoidance Through Neural Implicit Probabilistic Scenes

TL;DR

This work tackles safe robot navigation in NeRF-based environments by recasting NeRFs as Poisson Point Processes, enabling rigorous computation of collision probabilities. It introduces PURR, a voxelized map that fuses NeRF density with robot geometry, and a two-stage planner (A-based pathfinding plus Bézier trajectory optimization) that guarantees safety with a user-specified collision probability. The approach, dubbed CATNIPS, delivers real-time trajectory planning (around online) and outperforms prior NeRF-based planners in safety and conservatism, validated in both simulations and hardware. The PPP interpretation further offers a probabilistic perspective on NeRF training, uncertainty quantification, and active sensing, with broad implications for perception-guided planning and safe exploration in vision-based robotics.

Abstract

We introduce a transformation of a Neural Radiance Field (NeRF) to an equivalent Poisson Point Process (PPP). This PPP transformation allows for rigorous quantification of uncertainty in NeRFs, in particular, for computing collision probabilities for a robot navigating through a NeRF environment. The PPP is a generalization of a probabilistic occupancy grid to the continuous volume and is fundamental to the volumetric ray-tracing model underlying radiance fields. Building upon this PPP representation, we present a chance-constrained trajectory optimization method for safe robot navigation in NeRFs. Our method relies on a voxel representation called the Probabilistic Unsafe Robot Region (PURR) that spatially fuses the chance constraint with the NeRF model to facilitate fast trajectory optimization. We then combine a graph-based search with a spline-based trajectory optimization to yield robot trajectories through the NeRF that are guaranteed to satisfy a user-specific collision probability. We validate our chance constrained planning method through simulations and hardware experiments, showing superior performance compared to prior works on trajectory planning in NeRF environments. Our codebase can be found at https://github.com/chengine/catnips, and videos can be found on our project page (https://chengine.github.io/catnips).
Paper Structure (27 sections, 5 theorems, 32 equations, 10 figures, 1 table)

This paper contains 27 sections, 5 theorems, 32 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Planning and Control with Onboard Sensing and SDFs
  4. Planning and Control using NeRFs
  5. Other Uses of NeRFs in Robotics
  6. Chance-constrained Planning
  7. Neural Radiance Fields (NeRFs)
  8. NeRF Density as a Poission Point Process
  9. Poisson Point Processes
  10. "Rendering" a Marked PPP
  11. Equivalence of NeRF and PPP Rendering
  12. Discussion of PPP Interpretation
  13. Computing Collision Probability with NeRF Scenes
  14. Chance-Constrained Trajectory Generation in NeRFs
  15. Generating the PURR
  16. ...and 12 more sections

Key Result

Proposition 1

Consider a PPP $\lambda(\mathbf{x})$ and radiance $\mathbf{c}(\mathbf{x},\mathbf{d})$ and let the radiance satisfy (Ass1:Color) from Assumption Ass:Smooth. Then, the expected color of a pixel rendered from the PPP matches the form of the rendering equation (Eq:RayTracing).

Figures (10)

  • Figure 1: (a) Ground-truth of the Stonehenge scene, (b) Poisson Point Process (PPP) of the scene represented as a point cloud, (c) Probabilistically Unsafe Robot Region (PURR) of scene, (d) Generated safe paths from CATNIPS.
  • Figure 2: In the rendering process, the probability that the pixel color takes on the color of the infintesimally small occluding slice (green) is given by the probability that all slices in the region preceding the slice (red) are unoccluded. Then, the pixel color is the expectation of the color taken by varying the position of the occluding slice along the ray.
  • Figure 3: Overlay of a realization of the PPP with the ground-truth mesh of Stonehenge. The two have strong spatial agreement.
  • Figure 4: NeRF to PURR pipeline. (1) A density grid is sampled from the NeRF, which is then trilinearly interpolated and integrated over a particular voxel to retrieve the cell intensity grid. (2) A robot kernel is generated by taking the Minkowski sum between the minimal bounding sphere of the robot and a single voxel. (3) The kernel is used in a Conv3D operation with the cell intensity voxel grid to create the robot intensity grid, which we then threshold by the user-defined collision probability $\sigma$ to create the PURR.
  • Figure 5: Top: Mesh of Flightroom, and PURR with varying specified inter-penetration values $V_{max}$ at fixed probability $\sigma = 95\%$. Bottom: Density-thresholded voxel maps with varying density values $\rho$. The PURR is precisely calibrated to give a desired probability of collision, while thresholding the NeRF density directly offers no particular safety guarantee. The density map can quickly degenerate based on the threshold, while the PURR can still capture the geometry for reasonable ranges of $V_{max}$.
  • ...and 5 more figures

Theorems & Definitions (18)

  • Definition 1: Poisson Point Process
  • Proposition 1: Rendering of PPP
  • proof
  • Proposition 2: NeRF-PPP Equivalence
  • proof
  • Corollary 1
  • Definition 2: Probabilisitically Safe
  • Definition 3: Collision Offset Factor
  • Theorem 1
  • proof
  • ...and 8 more