RAnGE: Reachability Analysis for Guaranteed Ergodicity

TL;DR

RAnGE tackles guaranteeing coverage in disturbed environments by formulating ergodic exploration as a differential game between a controller and a worst-case disturbance and deriving an extended-state Hamilton-Jacobi-Isaacs (HJI) PDE. By transforming the ergodic metric into a Bolza-form objective with an augmented state $z$, the authors solve for a robust controller with a deep neural network-based reachability solver and compare against a robust MPC baseline. They introduce RAnGE and a complementary reMPC approach, validating robust ergodic trajectories on a 1D double-integrator and, in experiments, on a quadrotor under disturbance, while highlighting practical scalability limits. The work provides formal worst-case guarantees for ergodic exploration under disturbances and offers a path toward extending guarantees to higher-dimensional search spaces with improved solvers.

Abstract

This paper investigates performance guarantees on coverage-based ergodic exploration methods in environments containing disturbances. Ergodic exploration methods generate trajectories for autonomous robots such that time spent in each area of the exploration space is proportional to the utility of exploring in the area. We find that it is possible to use techniques from reachability analysis to solve for optimal controllers that guarantee ergodic coverage and are robust against disturbances. We formulate ergodic search as a differential game between the controller optimizing for ergodicity and an external disturbance, and we derive the reachability equations for ergodic search using an extended-state Bolza-form transform of the ergodic problem. Contributions include the computation of a continuous value function for the ergodic exploration problem and the derivation of a controller that provides guarantees for coverage under disturbances. Our approach leverages neural-network-based methods to solve the reachability equations; we also construct a robust model-predictive controller for comparison. Simulated and experimental results demonstrate the efficacy of our approach for generating robust ergodic trajectories for search and exploration on a 1D system with an external disturbance force.

Figures (5)

• Figure 1: Overview of RAnGE. An agent explores an area with distributed information, subject to a disturbance (a), using a precomputed value function (b) to achieve comprehensive ergodic coverage (c). Note that in (b), higher values denote less desirable states.
• Figure 2: Trajectories generated by RAnGE in simulation, with worst-case disturbance, for bimodal and uniform information distributions. For each pair of plots, the left plot shows the trajectory and the direction of the disturbance, and the right plot shows the coverage achieved by the trajectory.
• Figure 3: Time-Evolution of a Trajectory and the Value Function. Top row: the time-averaged spatial distribution and the information distribution. As time increases, the time-averaged trajectory statistics approach the information distribution, which is the goal of ergodic exploration. Bottom: evolution of the trajectory through phase space, superimposed on a cross-section of the value function with respect to position and velocity, where $z_k$ are fixed to their value at that instant in the trajectory.
• Figure 4: Comparison of MPC, reMPC, and RAnGE. The performance measures are the ergodic metric \ref{['Equivalent Metrics']} and cost function \ref{['Ergodic Bolza Cost']}. Note that the ergodic metric is not normalized; see Figure \ref{['fig:Trajectory with Value']} for an intuition of the scale of $\mathcal{E}$. The MPC and reMPC results aggregate 64 runs with varying initial conditions; the RAnGE results aggregate from 64 runs each with two controllers trained with different random seeds. The worst-case disturbance came from RAnGE according to \ref{['eq:range_disturbance']}, the uniform disturbance was sampled from $[-d_{\mathrm{max}},d_{\mathrm{max}}]$, and the normal disturbance was sampled from a Gaussian with $\mu=0,\sigma=d_{\mathrm{max}}/2$. All trials lasted for 20s, and all controllers had a 1s time horizon, so the running cost was divided by 20 when computing the trajectory cost $J$, to preserve the relative weighting of the running and terminal costs.
• Figure 5: Experimental setup and results. (a) We used a Bitcraze Crazyflie 2.1 quadrotor drone, and two Lighthouse cameras (not pictured) to assist with position and velocity measurement crazyswarm. The information distribution was a hard-coded bimodal distribution, and the blocks show the modes and bounds for visualization purposes only. Bottom row: trajectories recorded from the drone, with the fans (b) turned on at constant low speed, (c) turned off, and (d) turned on at varying speeds from low to high over the course of the trajectory.

Theorems & Definitions (13)

• definition thmcounterdefinition: Ergodicity
• definition thmcounterdefinition: Time-averaged trajectory statistics
• definition thmcounterdefinition: Spectral ergodic metric
• definition thmcounterdefinition: Value function
• definition thmcounterdefinition: Hamilton-Jacobi-Isaacs optimality conditions
• lemma thmcounterlemma: Augmented-state ergodic metric formulation mathew_metrics_2011de_la_torre_ergodic_2016
• proof
• theorem thmcountertheorem: Ergodic Hamilton-Jacobi-Isaac PDE
• proof
• remark thmcounterremark: Cost guarantees