Quality-Diversity Optimization as Multi-Objective Optimization

TL;DR

This paper reframes Quality-Diversity optimization as a many-objective optimization problem by densely covering the behavior space with a large set of objectives and optimizing a small, collaborative set of solutions. It introduces set-based scalarizations (SoM, TCH-Set) and their smooth counterparts (SSoM, STCH-Set) to enable gradient-based optimization without reliance on discrete archives, while establishing monotonicity, supermodularity, and Pareto-optimality guarantees. Empirically, the smooth MOO methods achieve competitive or superior QD performance across LP, IC, and LSI benchmarks, often surpassing state-of-the-art QD baselines in diversity-robustness metrics such as QVS. The work demonstrates a scalable, flexible framework for QD that leverages MOO tools and paves the way for extensions to non-differentiable settings and preference-guided search.

Abstract

The Quality-Diversity (QD) optimization aims to discover a collection of high-performing solutions that simultaneously exhibit diverse behaviors within a user-defined behavior space. This paradigm has stimulated significant research interest and demonstrated practical utility in domains including robot control, creative design, and adversarial sample generation. A variety of QD algorithms with distinct design principles have been proposed in recent years. Instead of proposing a new QD algorithm, this work introduces a novel reformulation by casting the QD optimization as a multi-objective optimization (MOO) problem with a huge number of optimization objectives. By establishing this connection, we enable the direct adoption of well-established MOO methods, particularly set-based scalarization techniques, to solve QD problems through a collaborative search process. We further provide a theoretical analysis demonstrating that our approach inherits theoretical guarantees from MOO while providing desirable properties for the QD optimization. Experimental studies across several QD applications confirm that our method achieves performance competitive with state-of-the-art QD algorithms.

Figures (5)

• Figure 1: Illustration of Different QD Optimization Algorithms.(a) MAP-Elitesmouret2015illuminating discretizes the behavior space into a fixed grid with some cells ($25$ in this case) and finds one local optimal solution per cell. (b) CVT-MAP-Elitesvassiliades2017using partitions the behavior space into a user-defined number of convex regions ($8$ in this case) and finds one local optimal solution per region. (c) Soft QDhedayatian2025soft uses several solutions ($8$ black dots in this case) to illuminate the whole behavior space without explicit partition. (d) QD as MOO (Ours) reformulates the QD optimization as an MOO problem with many optimization objectives ($200$ white crosses in this case) spanning the whole behavior space, and finds a smaller set of diverse solutions ($8$ black dots in this case) to tackle all optimization objectives in a collaborative manner.
• Figure 2: Performance on Linear Projection (LP) across $4$, $8$ and $16$-dimensional behavior space. Reported values are the mean and standard deviation over 10 independent runs. The smooth MOO-based methods, SSoM and STCH-Set, achieve the best QVS on both the Medium ($d=8$) and Hard ($d=16$) tasks. They maintain stable performance across all other comparisons, on par with SQUAD.
• Figure 3: Rastrigin Function
• Figure 4: Girl with a Pearl Earring
• Figure 5: Effect of Bandwidth $\gamma^2$

Theorems & Definitions (13)

• Definition 3.1: Dominance
• Definition 3.2: (Weakly) Pareto Optimal Solution
• Definition 3.3: Pareto Set and Pareto Front
• Theorem 3.4: Monotonicity
• Theorem 3.5: Supermodularity
• Theorem 3.6: Pareto Optimality of Solutions
• Theorem 3.7: Existence of Pareto Optimal Solution
• Theorem 3.8: Smooth Approximation
• proof
• proof