DR-MPC: Deep Residual Model Predictive Control for Real-world Social Navigation

TL;DR

DR-MPC introduces a deep residual framework that fuses MPC-based path tracking with a learned human-avoidance policy to enable real-world social navigation from limited data. By processing path-tracking and human-avoidance information in parallel and merging them via learned weighting, DR-MPC achieves near-MPC behavior initially and improved interaction with humans through training, aided by an OOD detector and heuristic safety checks. Real-world experiments demonstrate safe, efficient navigation with under 4 hours of training data, while simulations show clear advantages over naive DRL and residual DRL baselines and competitive performance relative to ORCA on safety and comfort metrics. This approach reduces the sim-to-real gap and provides a scalable framework for data-efficient, safe robot navigation in crowds.

Abstract

How can a robot safely navigate around people with complex motion patterns? Deep Reinforcement Learning (DRL) in simulation holds some promise, but much prior work relies on simulators that fail to capture the nuances of real human motion. Thus, we propose Deep Residual Model Predictive Control (DR-MPC) to enable robots to quickly and safely perform DRL from real-world crowd navigation data. By blending MPC with model-free DRL, DR-MPC overcomes the DRL challenges of large data requirements and unsafe initial behavior. DR-MPC is initialized with MPC-based path tracking, and gradually learns to interact more effectively with humans. To further accelerate learning, a safety component estimates out-of-distribution states to guide the robot away from likely collisions. In simulation, we show that DR-MPC substantially outperforms prior work, including traditional DRL and residual DRL models. Hardware experiments show our approach successfully enables a robot to navigate a variety of crowded situations with few errors using less than 4 hours of training data.

Figures (8)

• Figure 1: DR-MPC navigating in the real world. In this illustration, the robot deviates from its path to allow human 1 to pass and then slows down (red means slower speed) to let human 2 to pass before returning to its path.
• Figure 2: Full real-world pipeline. The Ouster OS0-128 LiDAR generates a detailed reflectivity image and a point cloud. The reflectivity image allows us to perform human tracking and the point cloud enables localization, path tracking, and depth recovery. With the state constructed from a single sensor, we use our OOD module and CVMM safety check to determine whether or not to execute the DR-MPC or the heuristic safety policy.
• Figure 3: DR-MPC architecture. The dark blue text elements involve learning. From $\mathcal{S}_{\text{PT}}$, we generate the MPC path tracking action and a latent embedding of the path information using an MLP. From $\mathcal{S}_{\text{HA}}$, we use a SOTA human avoidance network to generate six actions for human avoidance. The model then fuses all the information to generate $\boldsymbol{\alpha}$ and $\mathbf{p}$, which generates the final action to maximize the human avoidance and path tracking rewards.
• Figure 4: Simulation scenarios. The path is black, the corridors are red, the robot is yellow, and the humans are grey.
• Figure 5: Simulation results averaged over 10 trials. Both DR-MPC models outperform naïve DRL and Residual DRL by efficient task switching for human avoidance.