Neural NMPC through Signed Distance Field Encoding for Collision Avoidance

TL;DR

The paper presents SDF-NMPC, a system that encodes a single onboard range observation into a differentiable neural SDF and constrains a velocity-tracking NMPC to achieve mapless collision avoidance in unknown environments. It introduces a two-network architecture (a $eta$-VAE encoder and a coordinate-based MLP) to predict the SDF, trained in a sequential fashion, with a sampling-based scheme to generate targets. The NMPC treats the SDF as a differentiable constraint and enforces FoV limits, providing recursive feasibility and Lyapunov-like stability guarantees under fixed observations, while handling perception and odometry noise via slack variables. The approach is validated through extensive simulations and real-world forest experiments, demonstrating robust collision avoidance, resilience to odometry drift, and competitive performance against map-based and mapless baselines, with an open-source release for reproducibility and extension. The work advances mapless navigation by tightly integrating learned 3D environment representations with principled nonlinear control, enabling real-time, safety-critical operation in cluttered, unknown outdoor environments.

Abstract

This paper introduces a neural Nonlinear Model Predictive Control (NMPC) framework for mapless, collision-free navigation in unknown environments with Aerial Robots, using onboard range sensing. We leverage deep neural networks to encode a single range image, capturing all the available information about the environment, into a Signed Distance Function (SDF). The proposed neural architecture consists of two cascaded networks: a convolutional encoder that compresses the input image into a low-dimensional latent vector, and a Multi-Layer Perceptron that approximates the corresponding spatial SDF. This latter network parametrizes an explicit position constraint used for collision avoidance, which is embedded in a velocity-tracking NMPC that outputs thrust and attitude commands to the robot. First, a theoretical analysis of the contributed NMPC is conducted, verifying recursive feasibility and stability properties under fixed observations. Subsequently, we evaluate the open-loop performance of the learning-based components as well as the closed-loop performance of the controller in simulations and experiments. The simulation study includes an ablation study, comparisons with two state-of-the-art local navigation methods, and an assessment of the resilience to drifting odometry. The real-world experiments are conducted in forest environments, demonstrating that the neural NMPC effectively performs collision avoidance in cluttered settings against an adversarial reference velocity input and drifting position estimates.

Figures (16)

• Figure 1: An overview of the proposed SDF-NMPC method. The left side depicts the neural architecture used to approximate the mapping between the input depth images and the corresponding SDF, through sampling-based training in the 3D sensor frustum. The Convolutional encoder-decoder and MLP networks are trained sequentially. The right side presents the proposed control scheme, highlighting the contributed neural-constrained NMPC for velocity tracking, and the color-coded learning-based components.
• Figure 2: A 2D visualization of the distance transform (left) with two visible obstacles (gray). The blue line marks the $0$-level set of the SDF, and its dashed part illustrates its heuristic extension beyond the FoV $\mathbb{F}$ for training purposes. The zoom-in (right) illustrates the grid-based approximation of the distance transform for $\mathbf{p}$ by the distance $d$ between the central cell and the closest cell with a different occupancy (in red).
• Figure 3: Architecture of the neural networks. Convolution and deconvolution layers are pictured in orange, linear layers in green, activations in purple, and pooling layers in blue. The encoder computes a Gaussian latent representation of the image, parametrized by its mean and std. The latent is sampled and decoded to reconstruct the input image. The decoder (blue rectangle) is used for training the VAE but is disabled for inference. The Residual block for a given size $N$ is pictured in the gray rectangle and instantiated in the network structure. Residual Deconvolution blocks follow the exact same structure.
• Figure 4: Planar representation of the relevant 3D frames used in the paper, including the inertial frames at time $t_0$ where the sensor observation is captured (left), and the sensor frustum $\mathbb{F}$.
• Figure 5: Reconstruction error using the proposed biased VAE and a vanilla VAE. The blue pixels correspond to reconstructions "closer" than the actual pixel value. The green circles highlight instances of thin obstacles whose reconstructions are improved.