Infinite-Horizon Ergodic Control via Kernel Mean Embeddings

Abstract

This paper derives an infinite-horizon ergodic controller based on kernel mean embeddings for long-duration coverage tasks on general domains. While existing kernel-based ergodic control methods provide strong coverage guarantees on general coverage domains, their practical use has been limited to sub-ergodic, finite-time horizons due to intractable computational scaling, prohibiting its use for long-duration coverage. We resolve this scaling by deriving an infinite-horizon ergodic controller equipped with an extended kernel mean embedding error visitation state that recursively records state visitation. This extended state decouples past visitation from future control synthesis and expands ergodic control to infinite-time settings. In addition, we present a variation of the controller that operates on a receding-horizon control formulation with the extended error state. We demonstrate theoretical proof of asymptotic convergence of the derived controller and show preservation of ergodic coverage guarantees for a class of 2D and 3D coverage problems.

Figures (11)

• Figure 1: Infinite-Horizon Ergodic Control Feedback Loop. The desired target distribution is embedded into a kernel mean embedding function $\mu_q$ that is tracked. At each time-step, the agent computes an ergodic control based on its current state $x(t)$ and its visitation embedding $\mu_{\rho_{x,t}}$ according to its time-average visitation distribution $\rho_{x,t}$. The ergodic controller then seeks to minimize the distance between these embeddings $e_{\omega_i}(t)$ at distinct points in the domain $\omega_i$ as $t\to\infty$, inducing ergodicity.
• Figure 2: Time Series Evolution of Ergodic Controller \ref{['eq:ergodic_ctrl_law']}. Illustrated are time snapshots of the infinite-time ergodic controller derived form the kernel mean embedding for (top) a planar multi-modal target distribution and (bottom) a bunny manifold with uniform distribution. The dynamics are given by the single-integrator system and the error visitation state is tracked via \ref{['eq:ivp_error_state']} for $M=500$ samples. As $t\to \infty$, the ergodic controller fully covers the domain without increasing computation.
• Figure 3: Reduction of Ergodicity vs. Planning Horizon. Here, we illustrate the impact that the receding-horizon formulation in \ref{['eq:plannig_opt']} has on the reduction of ergodicity over the planning horizon for a single-integration dynamical system. As the planning horizon increases, so does the reduction in ergodicity. Note that for large samples, numerical precision becomes a limited factor which motivates the use of the extended error visitation state which is a direct tradeoff between planning controls versus reactive control.
• Figure 4: Impact of MPC Planning Horizon on Short-Term Ergodic Convergence. Here, we show a qualitative illustration of the impact of control planning horizon on the effective coverage trajectory. As the time-horizon increases, the trajectory tries to span more of domain proportional to the target distribution. However, as planning time becomes longer, so does the complexity of the trajectory and control, leading to numerical instabilities.
• Figure 5: Illustration of Infinite-Horizon Feedback Control. (Left) A target reference distribution $q$ on $\Omega$ with samples $\{ \omega_i \}_{i=0}^M \sim q$. (Right) An ergodic trajectory from a single-integrator dynamical system following the proposed control law \ref{['eq:ergodic_ctrl_law']}.

Theorems & Definitions (13)

• Definition 1
• Definition 2
• Theorem 1: Weak Convergence of MMD, simon2023, Theorem 7
• Lemma 1: Expanded MMD, gretton2012, Lemma 4
• Proposition 1: Ergodic MMD Metric
• proof
• Definition 3
• Proposition 2: Visitation Error State Representation of Ergodic MMD Metric
• proof
• Theorem 2: Infinite-Horizon Ergodic Control