Drivora: A Unified and Extensible Infrastructure for Search-based Autonomous Driving Testing

TL;DR

The paper tackles the challenge of evaluating ADS safety amid heterogeneous testing frameworks. It introduces Drivora, a CARLA-based, unified infrastructure that uses an OpenScenario low-level DSL and a decoupled architecture to enable scalable, multi-AV testing. Core contributions include a four-component tool design (ADS integration, testing configuration, testing engine, and Scenario Execution Layer), a parallel execution mechanism, and integration of twelve ADSs with five testing methods. Through demo results, Drivora demonstrates the ability to discover safety-critical violations and to achieve near-linear throughput gains with parallelism. The work provides a practical, open-source platform for researchers to prototype new testing methods without environment setup overhead.

Abstract

Search-based testing is critical for evaluating the safety and reliability of autonomous driving systems (ADSs). However, existing approaches are often built on heterogeneous frameworks (e.g., distinct scenario spaces, simulators, and ADSs), which require considerable effort to reuse and adapt across different settings. To address these challenges, we present Drivora, a unified and extensible infrastructure for search-based ADS testing built on the widely used CARLA simulator. Drivora introduces a unified scenario definition, OpenScenario, that specifies scenarios using low-level, actionable parameters to ensure compatibility with existing methods while supporting extensibility to new testing designs (e.g., multi-autonomous-vehicle testing). On top of this, Drivora decouples the testing engine, scenario execution, and ADS integration. The testing engine leverages evolutionary computation to explore new scenarios and supports flexible customization of core components. The scenario execution can run arbitrary scenarios using a parallel execution mechanism that maximizes hardware utilization for large-scale batch simulation. For ADS integration, Drivora provides access to 12 ADSs through a unified interface, streamlining configuration and simplifying the incorporation of new ADSs. Our tools are publicly available at https://github.com/MingfeiCheng/Drivora.

Figures (2)

• Figure 1: The overall architecture of the Drivora framework.
• Figure 2: Illustration of violation cases.