PAFOT: A Position-Based Approach for Finding Optimal Tests of Autonomous Vehicles
Victor Crespo-Rodriguez, Neelofar, Aldeida Aleti
TL;DR
PAFOT addresses the challenge of efficiently finding safety-critical scenarios for autonomous driving by introducing a position-based, 9-position grid around the Ego Vehicle and encoding NPC maneuvers as Position-Instructions. It employs a single-objective GA to optimize scenario risk (METTC, MD, SD) and execution time (ET), supplemented by a Local Fuzzer and a random Restart to enhance exploration. Empirical evaluation in CARLA shows PAFOT discovers more safety-critical scenarios and does so faster than AV-Fuzzer and Random baselines, reducing overall simulation time. The work highlights potential extensions to industrial ADS platforms and multi-objective optimization to increase scenario diversity and robustness.
Abstract
Autonomous Vehicles (AVs) are prone to revolutionise the transportation industry. However, they must be thoroughly tested to avoid safety violations. Simulation testing plays a crucial role in finding safety violations of Automated Driving Systems (ADSs). This paper proposes PAFOT, a position-based approach testing framework, which generates adversarial driving scenarios to expose safety violations of ADSs. We introduce a 9-position grid which is virtually drawn around the Ego Vehicle (EV) and modify the driving behaviours of Non-Playable Characters (NPCs) to move within this grid. PAFOT utilises a single-objective genetic algorithm to search for adversarial test scenarios. We demonstrate PAFOT on a well-known high-fidelity simulator, CARLA. The experimental results show that PAFOT can effectively generate safety-critical scenarios to crash ADSs and is able to find collisions in a short simulation time. Furthermore, it outperforms other search-based testing techniques by finding more safety-critical scenarios under the same driving conditions within less effective simulation time.