AuthSim: Towards Authentic and Effective Safety-critical Scenario Generation for Autonomous Driving Tests

TL;DR

This paper tackles the problem of generating authentic safety-critical scenarios for autonomous driving tests, addressing the prevalence of extreme and unrealistic collisions in prior adversarial methods. It introduces AuthSim, which combines a real-time three-layer relative safety region model (danger, boundary, and safety) with reinforcement learning to bias NPC behavior toward boundary regions, yielding more rational and natural adversarial interactions. The framework defines a novel scenario-criticality objective $J(\theta)$ that incorporates region-based overlaps and region-appearance probabilities, and models the NPC attack as an MDP $M=\langle S,A,P,R,\gamma\rangle$ with a 5-action policy and a three-layer reward. Experimental results on Highway-env show that AuthSim achieves longer cut-in distances and longer collision intervals, increases the proportion of meaningful collisions, and improves authenticity metrics compared with NADE, TTC, TTB, and DRAC, demonstrating substantial gains in both realism and efficiency for safety testing.

Abstract

Generating adversarial safety-critical scenarios is a pivotal method for testing autonomous driving systems, as it identifies potential weaknesses and enhances system robustness and reliability. However, existing approaches predominantly emphasize unrestricted collision scenarios, prompting non-player character (NPC) vehicles to attack the ego vehicle indiscriminately. These works overlook these scenarios' authenticity, rationality, and relevance, resulting in numerous extreme, contrived, and largely unrealistic collision events involving aggressive NPC vehicles. To rectify this issue, we propose a three-layer relative safety region model, which partitions the area based on danger levels and increases the likelihood of NPC vehicles entering relative boundary regions. This model directs NPC vehicles to engage in adversarial actions within relatively safe boundary regions, thereby augmenting the scenarios' authenticity. We introduce AuthSim, a comprehensive platform for generating authentic and effective safety-critical scenarios by integrating the three-layer relative safety region model with reinforcement learning. To our knowledge, this is the first attempt to address the authenticity and effectiveness of autonomous driving system test scenarios comprehensively. Extensive experiments demonstrate that AuthSim outperforms existing methods in generating effective safety-critical scenarios. Notably, AuthSim achieves a 5.25% improvement in average cut-in distance and a 27.12% enhancement in average collision interval time, while maintaining higher efficiency in generating effective safety-critical scenarios compared to existing methods. This underscores its significant advantage in producing authentic scenarios over current methodologies.

Figures (11)

• Figure 1: Examples of extreme and unreasonable collision scenarios for autonomous driving tests. (a) NPC side attacks; (b) NPC dangerous lane changes.
• Figure 2: Illustration of longitudinal danger, boundary, and safety distances between the ego and NPC vehicles: (a) Danger distance; (b) Boundary distance; (c) Safety distance.
• Figure 3: The three-layer relative safety region model.
• Figure 4: Different overlapping area and staying time of the NPC vehicle in different relative regions.
• Figure 5: Real and modified estimation distributions of NPC vehicles appearing in different regions.