MADR: MPC-guided Adversarial DeepReach

TL;DR

MADR tackles robust safety under worst-case disturbances and adversarial agents in high-dimensional robotic systems by marrying Hamilton–Jacobi reachability with adversarial MPC-guided supervision. It learns a single value function $V(x,t)$ via two SB-MPC datasets and gradient-informed adversarial rollouts, enabling near-ground-truth backward reachable tubes (BRTs) across 13D, 6D, and 20D dynamics. A pursuit-evasion filter further stabilizes long-horizon performance, and hardware demonstrations on TurtleBots, Crazyflie drones, and humanoid interactions validate practical safety and robustness gains. Overall, MADR bridges theory-driven reachability with scalable, data-driven control in safety-critical autonomous systems, offering strong robustness to disturbances and intelligent opponents with real-world applicability.

Abstract

Hamilton-Jacobi (HJ) Reachability offers a framework for generating safe value functions and policies in the face of adversarial disturbance, but is limited by the curse of dimensionality. Physics-informed deep learning is able to overcome this infeasibility, but itself suffers from slow and inaccurate convergence, primarily due to weak PDE gradients and the complexity of self-supervised learning. A few works, recently, have demonstrated that enriching the self-supervision process with regular supervision (based on the nature of the optimal control problem), greatly accelerates convergence and solution quality, however, these have been limited to single player problems and simple games. In this work, we introduce MADR: MPC-guided Adversarial DeepReach, a general framework to robustly approximate the two-player, zero-sum differential game value function. In doing so, MADR yields the corresponding optimal strategies for both players in zero-sum games as well as safe policies for worst-case robustness. We test MADR on a multitude of high-dimensional simulated and real robotic agents with varying dynamics and games, finding that our approach significantly out-performs state-of-the-art baselines in simulation and produces impressive results in hardware.

Figures (7)

• Figure 1: A graphical abstract of MADR and the robotic experiments. In this work, we propose enriching self-supervised learning of HJ-PDE's, i.e. DeepReach, with supervision given by the best sampled game rollout (top left), where the opponents policy is defined by the current value approximation (top right). We demonstrate this approach performs across games with general dynamics (bottom), namely with TurtleBots, Drones and Humanoid experiments.
• Figure 2: Graphical depiction of the MPC formulation combining both player's rollouts in the loss function for the proposed MPG-guided adversarial PINN training.
• Figure 3: Representative trajectory rollouts comparing MADR and ISAACS in the 13D quadrotor environment under wind disturbances. The solid lines correspond to MADR, while the dashed lines correspond to ISAACS.
• Figure 4: Distribution of actual–predicted cost differences and safe rates over 100,000 trajectories, comparing MADR and ISAACS based on their value functions. Negative values indicate overestimation of safety.
• Figure 5: A hardware demonstration of the drone vs. drone pursuit-evasion game with MADR. From left to right, four trajectory snapshots show the position of the evader (blue) and the pursuer (red) up to that time point with captures or out-of-bounds (OOB) marked by an 'x' (black). Both agents employ our proposed policy, with the pursuer using the follow-filtered augmentation.