Tournament Informed Adversarial Quality Diversity

TL;DR

This work extends Generational Adversarial MAP-Elites (GAME) by introducing tournament-informed task selection, enabling fair inter-variant comparisons and improved illumination of both sides in adversarial quality diversity (QD). It formalizes the adversarial QD problem, proposes Ranking and Pareto task-selection methods, and defines six measures (Win rate, ELO Score, Robustness, Coverage, Expertise, AQD-Score) to capture quality and diversity. Empirical results across Pong, Cat-and-mouse, and Pursuers-and-evaders show that Ranking consistently yields higher adversarial quality and diversity than the original Behavior or Random baselines, while Pareto is competitive but generally weaker. The findings highlight the value of adversarially informed task selection for deeper illumination and suggest paths to enhance sample efficiency and open-endedness in future work.

Abstract

Quality diversity (QD) is a branch of evolutionary computation that seeks high-quality and behaviorally diverse solutions to a problem. While adversarial problems are common, classical QD cannot be easily applied to them, as both the fitness and the behavior depend on the opposing solutions. Recently, Generational Adversarial MAP-Elites (GAME) has been proposed to coevolve both sides of an adversarial problem by alternating the execution of a multi-task QD algorithm against previous elites, called tasks. The original algorithm selects new tasks based on a behavioral criterion, which may lead to undesired dynamics due to inter-side dependencies. In addition, comparing sets of solutions cannot be done directly using classical QD measures due to side dependencies. In this paper, we (1) use an inter-variants tournament to compare the sets of solutions, ensuring a fair comparison, with 6 measures of quality and diversity, and (2) propose two tournament-informed task selection methods to promote higher quality and diversity at each generation. We evaluate the variants across three adversarial problems: Pong, a Cat-and-mouse game, and a Pursuers-and-evaders game. We show that the tournament-informed task selection method leads to higher adversarial quality and diversity. We hope that this work will help further advance adversarial quality diversity. Code, videos, and supplementary material are available at https://github.com/Timothee-ANNE/GAME_tournament_informed.

Figures (3)

• Figure 1: GAME is a coevolutionary QD algorithm that illuminates adversarial problems by alternating the execution of MTMB-ME anne2023multi on a set of tasks (i.e., fixed solutions from the opposing side) to encourage arms race dynamics. For example, in this illustration, the blue letters can represent strategies for a mouse to avoid a cat, and the red letters represent strategies for a cat to catch a mouse.
• Figure 2: To illuminate an adversarial problem, GAME selects elites from the previous generation and sets them as opposing tasks for the next generation, which should represent a diversity of challenges. (a) In its original version, GAME selects tasks using a behavioral criterion. (b) In this paper, we propose two new methods that use an adversarial criterion by being informed by a tournament between the previous tasks and all elites: (c) Ranking, which selects a set of solutions that present a diversity of challenges, the idea being that different challenges should creates different rankings for the opposing side, and (d) Pareto, which selects a Pareto Front of solutions by considering the current tasks as a multi-objective problem.
• Figure 3: Pretty (top) and one-frame (bottom) visualization of the adversarial problems. (a-b) Pong, (c-d) Cat-and-mouse, and (e-f) Pursuers-and-evaders. The one-frame visualizations are from duels between the two solutions with the highest ELO score from the inter-variant tournament. Videos of those duels are available in the supplementary material.