Algorithmic Scenario Generation as Quality Diversity Optimization

TL;DR

A general framework for solving this problem is presented, the insights that are gained from working on each component are described, and how integrating these components leads to the discovery of a diverse range of realistic and challenging scenarios that reveal previously unknown failures in deployed robotic systems interacting with people.

Abstract

The increasing complexity of robots and autonomous agents that interact with people highlights the critical need for approaches that systematically test them before deployment. This review paper presents a general framework for solving this problem, describes the insights that we have gained from working on each component of the framework, and shows how integrating these components leads to the discovery of a diverse range of realistic and challenging scenarios that reveal previously unknown failures in deployed robotic systems interacting with people.

Figures (13)

• Figure 1: The user gets distracted and forgets to empty the pot. Picking up the pot and attempting to place it on the stove would result in the robot dropping the pot with the boiling water. Our work on scenario generation can help avoid potentially catastrophic failures.
• Figure 2: QD and other types of optimization algorithms. The blue shading shows the desired result for each type.
• Figure 3: Algorithmic scenario generation framework.
• Figure 4: The solution ${\bm{\theta}}'$ is mapped to a cell $e$ in the archive based on the measure values $\bm{m}({\bm{\theta}}')$. The cell may be empty or occupied by an incumbent solution ${\bm{\theta}}_e$
• Figure 5: Archive of generated Mario levels.