An Invariant Information Geometric Method for High-Dimensional Online Optimization

TL;DR

The paper tackles high-dimensional online zeroth-order optimization by introducing InvIGO, an invariant information-geometric optimization framework, and instantiating it with a multi-dimensional Gaussian to produce SynCMA. It leverages an approximate objective based on $D_{KL}(q_{ heta^t}||p_{ heta})$ and a line-search mechanism to fully incorporate historical information, while preserving invariant updates and $O(HN)$ per-step complexity. Theoretical analysis clarifies invariance properties and connections to CMA-ES, and experiments across Mujoco, rover planning, and synthetic benchmarks demonstrate competitive, often superior, sample efficiency relative to leading Bayesian optimization and evolution strategies. The approach highlights the potential of property-oriented, history-aware evolution strategies for challenging high-dimensional online optimization problems.

Abstract

Sample efficiency is crucial in optimization, particularly in black-box scenarios characterized by expensive evaluations and zeroth-order feedback. When computing resources are plentiful, Bayesian optimization is often favored over evolution strategies. In this paper, we introduce a full invariance oriented evolution strategies algorithm, derived from its corresponding framework, that effectively rivals the leading Bayesian optimization method in tasks with dimensions at the upper limit of Bayesian capability. Specifically, we first build the framework InvIGO that fully incorporates historical information while retaining the full invariant and computational complexity. We then exemplify InvIGO on multi-dimensional Gaussian, which gives an invariant and scalable optimizer SynCMA . The theoretical behavior and advantages of our algorithm over other Gaussian-based evolution strategies are further analyzed. Finally, We benchmark SynCMA against leading algorithms in Bayesian optimization and evolution strategies on various high dimension tasks, in cluding Mujoco locomotion tasks, rover planning task and synthetic functions. In all scenarios, SynCMA demonstrates great competence, if not dominance, over other algorithms in sample efficiency, showing the underdeveloped potential of property oriented evolution strategies.

Figures (6)

• Figure 1: Optimization procedure for two high dimensional Mujoco locomotion tasks over 10 trials and rover planning task over 100 trails. Index of SynCMA indicate $\lambda_0$.
• Figure 2: Optimization procedure in 4 typical synthetic functions with dimension $n = 64$ over 20 trails considering all optimizers. Index of SynCMA indicate $\lambda_0$.
• Figure 3: Optimization procedure in 10 tests function with dimension $n = 32$ over 100 trails with 10000 evaluations.
• Figure 4: Optimization procedure in 10 tests function with dimension $n = 64$ over 100 trails with 50000 evaluations.
• Figure 5: Optimization procedure in 10 tests function with dimension $n = 128$ over 100 trails with 100000 evaluations.

Theorems & Definitions (9)

• definition 1: IGO complexity
• definition 2: Invariant property
• proposition 1
• theorem 1: Invariant for InvIGO
• proposition 2: Theorem 4.1 in akimoto2012theoretical
• proposition 3
• proof
• proof
• proof