Towards a Completeness Argumentation for Scenario Concepts

TL;DR

The paper tackles the challenge of proving that a scenario concept for scenario-based testing of automated driving functions is sufficiently complete to cover open real-world contexts. It introduces a formal, goal-structured ($GSN$) argumentation framework that differentiates completeness from coverage and prescribes evidence strategies (data-driven and knowledge-based) to support the claims. By applying the methodology to a concrete scenario concept (WEB23) and the inD data, it demonstrates how enveloping scenarios, a six-layer modeling approach, and base/focus scenarios can be integrated into a traceable completeness argument, with data-driven validation showing substantial coverage of observed base scenarios and parameter saturations. The work provides a structured pathway for safety assurance, enabling traceable completeness arguments, data-backed coverage assessments, and a clear roadmap for updating scenarios as driving contexts evolve, with implications for both simulation-based testing and real-world validation.

Abstract

Scenario-based testing has become a promising approach to overcome the complexity of real-world traffic for safety assurance of automated vehicles. Within scenario-based testing, a system under test is confronted with a set of predefined scenarios. This set shall ensure more efficient testing of an automated vehicle operating in an open context compared to real-world testing. However, the question arises if a scenario catalog can cover the open context sufficiently to allow an argumentation for sufficiently safe driving functions and how this can be proven. Within this paper, a methodology is proposed to argue a sufficient completeness of a scenario concept using a goal structured notation. Thereby, the distinction between completeness and coverage is discussed. For both, methods are proposed for a streamlined argumentation and regarding evidence. These methods are applied to a scenario concept and the inD dataset to prove the usability.

Figures (5)

• Figure 1: Application of completeness and coverage
• Figure 2: Argumentation strategy TinoBrade23
• Figure 3: Scenario concept structure
• Figure 4: Part of argumentation structure with goals (dark blue), counter hypothesis (light blue) and evidence strategies (white)
• Figure 5: Saturation of parameters in inD data inD