Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quality-Diversity Algorithms Can Provably Be Helpful for Optimization

Chao Qian, Ke Xue, Ren-Jian Wang

TL;DR

The paper addresses whether Quality-Diversity (QD) algorithms can assist optimization by rigorously comparing MAP-Elites with the standard $(\mu+1)$-EA on two NP-hard problems: monotone approximately submodular maximization with a size constraint and set cover. It demonstrates that MAP-Elites achieves asymptotically optimal polynomial-time approximation ratios for both problems (specifically $1-e^{-\gamma_{\min}}$, and $\ln m + 1$ respectively), while there exist instances where the $(\mu+1)$-EA requires exponential time to reach comparable performance. The results highlight how maintaining diverse high-quality solutions across a discretized behavior space provides stepping stones to better global solutions and helps avoid local optima. This establishes theoretical justification for the effectiveness of QD algorithms in optimization and suggests directions for comparing QD frameworks and understanding their broader practical impact.

Abstract

Quality-Diversity (QD) algorithms are a new type of Evolutionary Algorithms (EAs), aiming to find a set of high-performing, yet diverse solutions. They have found many successful applications in reinforcement learning and robotics, helping improve the robustness in complex environments. Furthermore, they often empirically find a better overall solution than traditional search algorithms which explicitly search for a single highest-performing solution. However, their theoretical analysis is far behind, leaving many fundamental questions unexplored. In this paper, we try to shed some light on the optimization ability of QD algorithms via rigorous running time analysis. By comparing the popular QD algorithm MAP-Elites with $(μ+1)$-EA (a typical EA focusing on finding better objective values only), we prove that on two NP-hard problem classes with wide applications, i.e., monotone approximately submodular maximization with a size constraint, and set cover, MAP-Elites can achieve the (asymptotically) optimal polynomial-time approximation ratio, while $(μ+1)$-EA requires exponential expected time on some instances. This provides theoretical justification for that QD algorithms can be helpful for optimization, and discloses that the simultaneous search for high-performing solutions with diverse behaviors can provide stepping stones to good overall solutions and help avoid local optima.

Quality-Diversity Algorithms Can Provably Be Helpful for Optimization

TL;DR

The paper addresses whether Quality-Diversity (QD) algorithms can assist optimization by rigorously comparing MAP-Elites with the standard -EA on two NP-hard problems: monotone approximately submodular maximization with a size constraint and set cover. It demonstrates that MAP-Elites achieves asymptotically optimal polynomial-time approximation ratios for both problems (specifically , and respectively), while there exist instances where the -EA requires exponential time to reach comparable performance. The results highlight how maintaining diverse high-quality solutions across a discretized behavior space provides stepping stones to better global solutions and helps avoid local optima. This establishes theoretical justification for the effectiveness of QD algorithms in optimization and suggests directions for comparing QD frameworks and understanding their broader practical impact.

Abstract

Quality-Diversity (QD) algorithms are a new type of Evolutionary Algorithms (EAs), aiming to find a set of high-performing, yet diverse solutions. They have found many successful applications in reinforcement learning and robotics, helping improve the robustness in complex environments. Furthermore, they often empirically find a better overall solution than traditional search algorithms which explicitly search for a single highest-performing solution. However, their theoretical analysis is far behind, leaving many fundamental questions unexplored. In this paper, we try to shed some light on the optimization ability of QD algorithms via rigorous running time analysis. By comparing the popular QD algorithm MAP-Elites with -EA (a typical EA focusing on finding better objective values only), we prove that on two NP-hard problem classes with wide applications, i.e., monotone approximately submodular maximization with a size constraint, and set cover, MAP-Elites can achieve the (asymptotically) optimal polynomial-time approximation ratio, while -EA requires exponential expected time on some instances. This provides theoretical justification for that QD algorithms can be helpful for optimization, and discloses that the simultaneous search for high-performing solutions with diverse behaviors can provide stepping stones to good overall solutions and help avoid local optima.
Paper Structure (12 sections, 6 theorems, 22 equations, 1 figure, 2 algorithms)

This paper contains 12 sections, 6 theorems, 22 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. MAP-Elites Algorithm
  4. $(\mu+1)$-EA
  5. Algorithm Settings and Performance Measurement
  6. Monotone Approximately Submodular Maximization with a Size Constraint
  7. Analysis of MAP-Elites
  8. Analysis of $(\mu+1)$-EA
  9. Set Cover
  10. Analysis of MAP-Elites
  11. Analysis of $(\mu+1)$-EA
  12. Conclusion and Discussion

Key Result

Theorem 1

For maximizing a monotone approximately submodular function $f$ with a size constraint $k$, the expected running time of MAP-Elites with the parameter $I=n+1$, until finding a solution $\bm{x}$ with $|\bm{x}| = k$ and is $O(n^2(\log n+k))$, where $\gamma_{\min}=\min_{\bm{x}:|\bm{x}|=k-1}\gamma_{\bm{x},k}$, $\gamma_{\bm{x},k}$ is the submodularity ratio of $f$ w.r.t. $\bm{x}$ and $k$ as in Definit

Figures (1)

  • Figure 1: Two complete bipartite graphs.

Theorems & Definitions (15)

  • Definition 1: Submodularity Ratio das2011submodular
  • Definition 2: Monotone Approximately Submodular Function Maximization with a Size Constraint
  • Theorem 1
  • Lemma 1: Lemma 1 of qian2016parallel
  • Corollary 1
  • Definition 3: Maximum Coverage
  • Example 1
  • Theorem 2
  • proof
  • Definition 4: Set Cover
  • ...and 5 more