Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Determining the Tactical Challenge of Scenarios to Efficiently Test Automated Driving Systems

Lennart Vater, Sven Tarlowski, Michael Schuldes, Lutz Eckstein

TL;DR

The paper tackles the challenge of selecting relevant test scenarios for automated driving system safety validation by introducing the Challenge Description Method, which uses reachability analysis to describe the tactical difficulty of scenarios in terms of the minimum required lane changes and their timing. By computing a drivable area and constructing a lane-aware reachability graph, it yields human-interpretable descriptions that capture the necessary maneuvers without heavy metric calibration. The method supports offline pre-processing, maps scenarios into a CommonRoad-based framework, and uses forward/backward graph searches to determine the easiest and earliest feasible lane-change plans, producing concise descriptions of scenario difficulty. Four highway scenarios demonstrate that the approach handles both static and moving obstacles and provides insights beyond traditional per-scene metrics, enabling efficient filtering and selection of relevant test scenarios for multiple ADS configurations. The work thus offers a scalable, interpretable, and ADS-agnostic tool for scenario-based safety validation with practical impact on how test databases are curated and used in industry.

Abstract

The selection of relevant test scenarios for the scenario-based testing and safety validation of automated driving systems (ADSs) remains challenging. An important aspect of the relevance of a scenario is the challenge it poses for an ADS. Existing methods for calculating the challenge of a scenario aim to express the challenge in terms of a metric value. Metric values are useful to select the least or most challenging scenario. However, they fail to provide human-interpretable information on the cause of the challenge which is critical information for the efficient selection of relevant test scenarios. Therefore, this paper presents the Challenge Description Method that mitigates this issue by analyzing scenarios and providing a description of their challenge in terms of the minimum required lane changes and their difficulty. Applying the method to different highway scenarios showed that it is capable of analyzing complex scenarios and providing easy-to-understand descriptions that can be used to select relevant test scenarios.

Determining the Tactical Challenge of Scenarios to Efficiently Test Automated Driving Systems

TL;DR

The paper tackles the challenge of selecting relevant test scenarios for automated driving system safety validation by introducing the Challenge Description Method, which uses reachability analysis to describe the tactical difficulty of scenarios in terms of the minimum required lane changes and their timing. By computing a drivable area and constructing a lane-aware reachability graph, it yields human-interpretable descriptions that capture the necessary maneuvers without heavy metric calibration. The method supports offline pre-processing, maps scenarios into a CommonRoad-based framework, and uses forward/backward graph searches to determine the easiest and earliest feasible lane-change plans, producing concise descriptions of scenario difficulty. Four highway scenarios demonstrate that the approach handles both static and moving obstacles and provides insights beyond traditional per-scene metrics, enabling efficient filtering and selection of relevant test scenarios for multiple ADS configurations. The work thus offers a scalable, interpretable, and ADS-agnostic tool for scenario-based safety validation with practical impact on how test databases are curated and used in industry.

Abstract

The selection of relevant test scenarios for the scenario-based testing and safety validation of automated driving systems (ADSs) remains challenging. An important aspect of the relevance of a scenario is the challenge it poses for an ADS. Existing methods for calculating the challenge of a scenario aim to express the challenge in terms of a metric value. Metric values are useful to select the least or most challenging scenario. However, they fail to provide human-interpretable information on the cause of the challenge which is critical information for the efficient selection of relevant test scenarios. Therefore, this paper presents the Challenge Description Method that mitigates this issue by analyzing scenarios and providing a description of their challenge in terms of the minimum required lane changes and their difficulty. Applying the method to different highway scenarios showed that it is capable of analyzing complex scenarios and providing easy-to-understand descriptions that can be used to select relevant test scenarios.
Paper Structure (16 sections, 4 equations, 3 figures, 2 tables)

This paper contains 16 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. BACKGROUND AND DEFINITIONS
  4. Test Scenarios
  5. Tactical Challenge of Scenarios
  6. Reachability Analysis
  7. METHOD
  8. Design
  9. Reachability Analysis
  10. Lane-aware Reachability Graph
  11. Graph Search and Filtering
  12. EVALUATION
  13. Scenarios
  14. Challenge Description
  15. Discussion and Outlook
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Integration of the proposed Challenge Description Method into the scenario-based testing and safety validation approach. The pre-processing step (blue) is performed only once for multiple similar ADS and the selection and testing steps (orange) once per ADS.
  • Figure 2: Visualization of the lane-aware reachability graph (right) for base sets projected on the position domain (left).
  • Figure 3: Visualization of the challenge description of the four evaluation scenarios. Each lane has a width of $3.75~m$ in t-direction. Only each fifth time step of the moving vehicles is plotted.