Multi-Objective Quality-Diversity in Unstructured and Unbounded Spaces

TL;DR

This paper addresses the limitation of grid-based MOQD methods in unstructured and unbounded feature spaces by introducing MOUR-QD, a Multi-Objective Quality-Diversity algorithm that maintains a continuous, unstructured archive. It defines local Pareto fronts within a radius $l$ and proves improvements extend to a region of radius $2l$, enabling effective trade-off exploration without predefined feature bounds. Empirically, MOUR-QD matches or surpasses grid-based baselines on traditional MOQD tasks, demonstrates robustness when feature-space bounds are unknown, and enables unsupervised feature learning in latent spaces, opening doors to domains like protein design and latent exploration. The method shows strong performance across five robotic tasks and is backed by publicly available, containerised code for reproducibility and broader adoption.

Abstract

Quality-Diversity algorithms are powerful tools for discovering diverse, high-performing solutions. Recently, Multi-Objective Quality-Diversity (MOQD) extends QD to problems with several objectives while preserving solution diversity. MOQD has shown promise in fields such as robotics and materials science, where finding trade-offs between competing objectives like energy efficiency and speed, or material properties is essential. However, existing methods in MOQD rely on tessellating the feature space into a grid structure, which prevents their application in domains where feature spaces are unknown or must be learned, such as complex biological systems or latent exploration tasks. In this work, we introduce Multi-Objective Unstructured Repertoire for Quality-Diversity (MOUR-QD), a MOQD algorithm designed for unstructured and unbounded feature spaces. We evaluate MOUR-QD on five robotic tasks. Importantly, we show that our method excels in tasks where features must be learned, paving the way for applying MOQD to unsupervised domains. We also demonstrate that MOUR-QD is advantageous in domains with unbounded feature spaces, outperforming existing grid-based methods. Finally, we demonstrate that MOUR-QD is competitive with established MOQD methods on existing MOQD tasks and achieves double the MOQD-score in some environments. MOUR-QD opens up new opportunities for MOQD in domains like protein design and image generation.

Figures (6)

• Figure 1: Left: Existing moqd methods store Pareto Fronts in each cell of a MAP-Elites grid. Right:mour-qd introduces an unstructured archive, storing solutions in a continuous manner throughout the feature space. Pareto Fronts are defined locally, using a radius $2l$ around each solution.
• Figure 2: Two sets of solutions forming two Pareto Fronts. The hypervolume is reflected by the shaded areas in the objective space between the solutions and the front. The outer set of solutions achieve higher in both objectives and thus this front has a higher hypervolume.
• Figure 3: a) In mour-qd solutions are stored in a continuous, unstructured manner. b) When adding a solution to the archive, it is compared with other solutions that lie within a radius $l$ in the feature space. Solutions that offer a different trade-off are kept, but solutions that are dominated by the new solution are removed. c) Removing dominated solutions can negatively impact the set of possible trade-offs within $l$ of nearby solutions, but it guarantees an improved set of trade-offs within $2l$ of them.
• Figure 4: Performance of mour-qd compared to all other baselines. The line shows the median score and the shaded region shows the interquartile range across 10 seeds. In unsupervised tasks, mome provides an approximate upper bound for performance, rather than a comparative baseline.
• Figure 5: Final repertoire plots from the median run of each algorithm. Only tasks with 2-dimensional features are shown.

Theorems & Definitions (1)

• theorem 1