Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios

TL;DR

This work tackles the challenge of evaluating safety and robustness for autonomous driving in safety-critical scenarios, where edge cases are underrepresented in standard benchmarks. It introduces the SSAD safety testing platform and the SSDA evaluation framework to assess both perception-level robustness and system-level driving performance across static and dynamic scenarios, including adversarial attacks, distribution shifts, and accident-prone generation. The paper provides extensive AI-component and system-level experiments (digital/physical attacks, nuScenes/NuScenes-N, CARLA Town02/Town05) that reveal notable vulnerabilities and underscore the need for standardized, controllable, and diverse safety-critical scenarios. By offering a unified testing bed that bridges perception and control, the work aims to reduce real-world risk and accelerate safe deployment of autonomous driving technologies.

Abstract

Autonomous driving has made significant progress in both academia and industry, including performance improvements in perception task and the development of end-to-end autonomous driving systems. However, the safety and robustness assessment of autonomous driving has not received sufficient attention. Current evaluations of autonomous driving are typically conducted in natural driving scenarios. However, many accidents often occur in edge cases, also known as safety-critical scenarios. These safety-critical scenarios are difficult to collect, and there is currently no clear definition of what constitutes a safety-critical scenario. In this work, we explore the safety and robustness of autonomous driving in safety-critical scenarios. First, we provide a definition of safety-critical scenarios, including static traffic scenarios such as adversarial attack scenarios and natural distribution shifts, as well as dynamic traffic scenarios such as accident scenarios. Then, we develop an autonomous driving safety testing platform to comprehensively evaluate autonomous driving systems, encompassing not only the assessment of perception modules but also system-level evaluations. Our work systematically constructs a safety verification process for autonomous driving, providing technical support for the industry to establish standardized test framework and reduce risks in real-world road deployment.

Figures (9)

• Figure 1: Safety and robustness evaluation on safety-critical scenarios including the evaluation of AI component-level such as object detection and the evaluation of the functional safety of autonomous driving system.
• Figure 2: The categorization of safety-critical scenario includes static and dynamic traffic scenarios. static traffic scenario includes adversarial attacks and natural distributional shift. Dynamic traffic scenario is mainly accident-prone scenarios.
• Figure 3: The process of physical attack methods on target vehicles.
• Figure 4: Some real examples of natural distribution shift. Images are from the CODA dataset li2022coda.
• Figure 5: mAP and NDS v.s attack iterations.