Dynamic Neural Potential Field: Online Trajectory Optimization in the Presence of Moving Obstacles

TL;DR

Dynamic NPField-GPT introduces a Transformer-based predictor of footprint-aware collision costs that are differentiable and used by MPC to navigate dynamic indoor environments. It couples an autoregressive Transformer (NPField-GPT) with an SQP-based MPC via L4CasADi to produce real-time, constraint-aware trajectories expressed through the neural potential $J_o$. The study compares NPField-GPT with two baselines and standard planners (MPPI, CIAO*) in BenchMR and real Husky experiments, showing improved safety margins and robust behavior under motion changes, at the cost of higher computation time for GPT. The work provides a practical, open-source framework for learning-enabled planning that preserves model-based planning transparency while learning only the spatiotemporal collision risk.

Abstract

Generalist robot policies must operate safely and reliably in everyday human environments such as homes, offices, and warehouses, where people and objects move unpredictably. We present Dynamic Neural Potential Field (NPField-GPT), a learning-enhanced model predictive control (MPC) framework that couples classical optimization with a Transformer-based predictor of footprint-aware repulsive potentials. Given an occupancy sub-map, robot footprint, and optional dynamic-obstacle cues, our autoregressive NPField-GPT head forecasts a horizon of differentiable potentials that are injected into a sequential quadratic MPC program via L4CasADi, yielding real-time, constraint-aware trajectory optimization. We additionally study two baselines: (NPField-D1) static-frame decomposition and (NPField-D2) parallel MLP heads for all steps. In dynamic indoor scenarios from BenchMR and on a Husky UGV in office corridors, NPField-GPT produces safer, more conservative trajectories under motion changes, while D1/D2 offer lower latency. We also compare with the CIAO* and MPPI baselines. Across methods, the Transformer+MPC synergy preserves the transparency and stability of model-based planning while learning only the part that benefits from data: spatiotemporal collision risk. Code and trained models are available at https://github.com/CognitiveAISystems/Dynamic-Neural-Potential-Field

Figures (6)

• Figure 1: Overview of NPField-GPT integrated with MPC for dynamic obstacle avoidance. Left: Husky UGV operating in an office corridor. Middle: task window with occupancy grid, robot footprint, and a moving obstacle. Right: inputs—obstacle map, footprint, and dynamic-obstacle state $\left(x_{dyn}, y_{dyn}, \theta_{dyn}\right)$—are encoded and fed to NPField-GPT to predict a footprint-aware repulsive potential $J_o$. The potential is consumed inside the optimization loop by the MPC solver to compute $\mathbf{x}_{opt}[k\ldots k{+}m]$ and $\mathbf{u}_{opt}[k\ldots k{+}m]$, which are executed on the robot.
• Figure 2: NPField-D1 (left), NPField-D2 (middle), and NPField-GPT (right). Yellow: encoder (map and footprint); green: per-sample potential predictor; red: auxiliary decoder used only for training. D2 uses parallel heads for $t_0,\dots,t_{10}$, whereas NPField-GPT is an autoregressive transformer-based model that conditions $t{+}1$ on all prior outputs.
• Figure 3: Trajectory predictions for dynamic obstacles generated by NPField-D2 (middle row) and NPField-GPT (bottom row) under three distinct scenarios. Each column corresponds to a scenario, with the first, eighth, and tenth time steps visualized. The models predict repulsive potentials using only the initial map configuration (top row), while subsequent steps are autoregressively generated by NPField-GPT or predicted in parallel by D2 (via MLP heads). The rightmost scenario tests robustness by reversing the dynamic obstacle’s direction ($I_{dyn}$) relative to the validation dataset. Visualizations depict potentials across the full map for clarity, though the models inherently operate on per-query point-wise predictions.
• Figure 4: Illustration of a dynamic obstacle navigation scenario using the NPField-GPT architecture. The left panels depict the initial environment (left) and the corresponding neural potential field (right), while the right panels show the final state after trajectory execution. NPField-GPT autonomously predicts obstacle motion based solely on the initial map, without requiring explicit obstacle trajectory inputs, demonstrating self-contained dynamic collision avoidance capabilities.
• Figure 5: Husky UGV mobile manipulator