Dynamic Quality-Diversity Search

TL;DR

The paper tackles dynamic optimization challenges by introducing Dynamic Quality-Diversity (D-QD), a framework that adapts QD methods to time-varying environments. By coupling environment-shift detection, dynamic offspring generation, and selective archive re-evaluation, the authors instantiate Dynamic MAP-Elites and Dynamic CMA-ME and evaluate them on two dynamic benchmarks (Dynamic Sphere and Dynamic Lunar Lander). Across experiments, D-QD variants consistently outperform a no-update baseline and approach the performance of an all-updating oracle at a fraction of the re-evaluations required, though no single configuration is universally best. The work advances practical dynamic search by preserving diverse, high-quality archives while mitigating the computational burden of full re-evaluation, paving the way for robust QD in real-world, shifting domains.

Abstract

Evolutionary search via the quality-diversity (QD) paradigm can discover highly performing solutions in different behavioural niches, showing considerable potential in complex real-world scenarios such as evolutionary robotics. Yet most QD methods only tackle static tasks that are fixed over time, which is rarely the case in the real world. Unlike noisy environments, where the fitness of an individual changes slightly at every evaluation, dynamic environments simulate tasks where external factors at unknown and irregular intervals alter the performance of the individual with a severity that is unknown a priori. Literature on optimisation in dynamic environments is extensive, yet such environments have not been explored in the context of QD search. This paper introduces a novel and generalisable Dynamic QD methodology that aims to keep the archive of past solutions updated in the case of environment changes. Secondly, we present a novel characterisation of dynamic environments that can be easily applied to well-known benchmarks, with minor interventions to move them from a static task to a dynamic one. Our Dynamic QD intervention is applied on MAP-Elites and CMA-ME, two powerful QD algorithms, and we test the dynamic variants on different dynamic tasks.

Figures (2)

• Figure 1: The sphere environment, before a shift (left) and after (right).
• Figure 2: The complete trajectory of the same lander in the lunar lander environment, before a shift (blue) and after (red).