Discount Model Search for Quality Diversity Optimization in High-Dimensional Measure Spaces

TL;DR

This paper tackles the challenge of exploring high‑dimensional, distorted measure spaces in quality diversity (QD) optimization. It replaces the discrete, histogram discount of CMA‑MAE with a continuous neural discount model, enabling smoother, more informative improvement signals and sustained exploration. The proposed Discount Model Search (DMS) demonstrates strong gains over baseline QD methods on standard benchmarks and introduces QDDM, a setting where measures are specified via data such as images. The work broadens QD applicability to vision and art domains and highlights the tradeoffs between exploration, computational cost, and discount model reliability.

Abstract

Quality diversity (QD) optimization searches for a collection of solutions that optimize an objective while attaining diverse outputs of a user-specified, vector-valued measure function. Contemporary QD algorithms focus on low-dimensional measures because high-dimensional measures are prone to distortion, where many solutions found by the QD algorithm map to similar measures. For example, the CMA-MAE algorithm guides measure space exploration with a histogram in measure space that records so-called discount values. However, CMA-MAE stagnates in domains with high-dimensional measure spaces because solutions with similar measures fall into the same histogram cell and thus receive identical discount values. To address these limitations, we propose Discount Model Search (DMS), which guides exploration with a model that provides a smooth, continuous representation of discount values. In high-dimensional measure spaces, this model enables DMS to distinguish between solutions with similar measures and thus continue exploration. We show that DMS facilitates new QD applications by introducing two domains where the measure space is the high-dimensional space of images, which enables users to specify their desired measures by providing a dataset of images rather than hand-designing the measure function. Results in these domains and on high-dimensional benchmarks show that DMS outperforms CMA-MAE and other black-box QD algorithms.

Figures (12)

• Figure 1: (a): One failure mode of CMA-MAE. On a flat objective $f$, solutions ${\bm{\theta}}_1$ and ${\bm{\theta}}_2$ fall in the same archive cell based on their measures, resulting in identical discount values from the discount function $f_A$. (b): In our proposed DMS, the discount model provides a smooth discount function that assigns distinct discount values to ${\bm{\theta}}_1$ and ${\bm{\theta}}_2$, showing that ${\bm{\theta}}_2$ has greater archive improvement than ${\bm{\theta}}_1$ ($\Delta_2 > \Delta_1$) and thus providing a stronger signal to guide search. (c): Number of unique cells where solutions sampled by CMA-MAE land in two benchmarks (mean over 20 trials; \ref{['sec:distortion']}).
• Figure 2: In the LSI (Hiker) domain, the objective is "A photo of the face of a hiker," and the measure space is the space of images. We specify desired measures with landscape images from LHQ lhq. Thus, DMS finds images depicting what a hiker might look like in each landscape: hikers in thick jackets for the mountains or lighter clothing for the beach, and even a baby bundled up for the snow. Each hiker is shown to the left of their corresponding landscape.
• Figure 3: Mean and standard error of the mean for QD Score and Coverage of each algorithm in each domain. Standard error may not be visible in some plots.
• Figure 4: Mean and standard error of the mean for QD Score and Coverage of each algorithm in each domain. Standard error may not be visible in some plots.
• Figure 5: A random subset of images generated by DMS in the TA (MNIST) domain, where desired measures are sampled from the MNIST dataset. The goal in this domain is to arrange triangles to look like the given MNIST digits. Each rendered triangle image is shown to the left of its corresponding MNIST digit.