Dominated Novelty Search: Rethinking Local Competition in Quality-Diversity

TL;DR

This work reframes Quality-Diversity as a Genetic Algorithm with local competition implemented via fitness transformations, eliminating the need for predefined descriptor bounds, grids, or fixed distance thresholds. It introduces Dominated Novelty Search, which uses a dominated novelty-based competition fitness to balance fitness progression and behavioral diversity in a dynamic, space-adaptive manner. Empirical results across continuous control and maze tasks show DNS achieving state-of-the-art QD scores and robust performance in high-dimensional and unsupervised descriptor spaces. The approach provides a unified theoretical framework for local competition in QD and suggests directions for learning competition functions to further improve search efficiency and diversity.

Abstract

Quality-Diversity is a family of evolutionary algorithms that generate diverse, high-performing solutions through local competition principles inspired by natural evolution. While research has focused on improving specific aspects of Quality-Diversity algorithms, surprisingly little attention has been paid to investigating alternative formulations of local competition itself -- the core mechanism distinguishing Quality-Diversity from traditional evolutionary algorithms. Most approaches implement local competition through explicit collection mechanisms like fixed grids or unstructured archives, imposing artificial constraints that require predefined bounds or hard-to-tune parameters. We show that Quality-Diversity methods can be reformulated as Genetic Algorithms where local competition occurs through fitness transformations rather than explicit collection mechanisms. Building on this insight, we introduce Dominated Novelty Search, a Quality-Diversity algorithm that implements local competition through dynamic fitness transformations, eliminating the need for predefined bounds or parameters. Our experiments show that Dominated Novelty Search significantly outperforms existing approaches across standard Quality-Diversity benchmarks, while maintaining its advantage in challenging scenarios like high-dimensional and unsupervised spaces.

Figures (6)

• Figure 1: The descriptor specification problem. When trying to categorize the correct bounds for descriptors in QD optimization, one risks missing out on many quality-diverse solutions.
• Figure 2: Scatter Plots of Kheperax ($n=30$), a high dimensional domain, where a solution's behavior descriptor is $30$ evenly spaced positions throughout the agent's trajectory. Due to this, a MAP-Elites grid would need to be 60 dimensions, with many dimension combinations being infeasible (e.g., being at the bottom left at the first step, and at the top right at the next)
• Figure 3: Mujoco Environment Results. Each line represents the mean, with shaded regions representing 95% confidence intervals. These results are aggregated across 10 independent runs. Each column represents an environment, and each row represents a given metric after projection into a random grid.
• Figure 4: Kheperax with Varied Descriptor Dimensionality
• Figure 5: Kheperax with Unsupervised Behavior Descriptors