Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Concept for Semi-Automatic Configuration of Sufficiently Valid Simulation Setups for Automated Driving Systems

Niklas Braun, Markus Steimle, Martin Törngren, Markus Maurer

TL;DR

The paper tackles credibility challenges in simulation-based testing for automated driving systems by formalizing simulation model validity domains through design contracts. It introduces a semi automatic framework where simulation models and test cases are described by assume guaranteed contracts, enabling compositional configuration of simulation setups and the generation of runtime monitors. Key contributions include a formal contract theory tailored to simulation, definitions of simulation model validity domains and test case contracts, a suite of contract operations for composition, and a framework for practical configuration and monitoring. The approach aims to improve transparency, scalability, and credibility by separating concerns between model developers and users and by guiding the selection of cost effective yet sufficiently valid simulation configurations.

Abstract

As simulation is increasingly used in scenario-based approaches to test Automated Driving Systems, the credibility of simulation results is a major concern. Arguably, credibility depends on the validity of the simulation setup and simulation models. When selecting appropriate simulation models, a trade-off must be made between validity, often connected to the model's fidelity, and cost of computation. However, due to the large number of test cases, expert-based methods to create sufficiently valid simulation setups seem infeasible. We propose using design contracts in order to semi-automatically compose simulation setups for given test cases from simulation models and to derive requirements for the simulation models, supporting separation of concerns between simulation model developers and users. Simulation model contracts represent their validity domains by capturing a validity guarantee and the associated operating conditions in an assumption. We then require the composition of the simulation model contracts to refine a test case contract. The latter contract captures the operating conditions of the test case in its assumption and validity requirements in its guarantee. Based on this idea, we present a framework that supports the compositional configuration of simulation setups based on the contracts and a method to derive runtime monitors for these simulation setups.

A Concept for Semi-Automatic Configuration of Sufficiently Valid Simulation Setups for Automated Driving Systems

TL;DR

The paper tackles credibility challenges in simulation-based testing for automated driving systems by formalizing simulation model validity domains through design contracts. It introduces a semi automatic framework where simulation models and test cases are described by assume guaranteed contracts, enabling compositional configuration of simulation setups and the generation of runtime monitors. Key contributions include a formal contract theory tailored to simulation, definitions of simulation model validity domains and test case contracts, a suite of contract operations for composition, and a framework for practical configuration and monitoring. The approach aims to improve transparency, scalability, and credibility by separating concerns between model developers and users and by guiding the selection of cost effective yet sufficiently valid simulation configurations.

Abstract

As simulation is increasingly used in scenario-based approaches to test Automated Driving Systems, the credibility of simulation results is a major concern. Arguably, credibility depends on the validity of the simulation setup and simulation models. When selecting appropriate simulation models, a trade-off must be made between validity, often connected to the model's fidelity, and cost of computation. However, due to the large number of test cases, expert-based methods to create sufficiently valid simulation setups seem infeasible. We propose using design contracts in order to semi-automatically compose simulation setups for given test cases from simulation models and to derive requirements for the simulation models, supporting separation of concerns between simulation model developers and users. Simulation model contracts represent their validity domains by capturing a validity guarantee and the associated operating conditions in an assumption. We then require the composition of the simulation model contracts to refine a test case contract. The latter contract captures the operating conditions of the test case in its assumption and validity requirements in its guarantee. Based on this idea, we present a framework that supports the compositional configuration of simulation setups based on the contracts and a method to derive runtime monitors for these simulation setups.
Paper Structure (32 sections, 3 figures)

This paper contains 32 sections, 3 figures.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Scenario-Based Testing
  4. Simulation-Based Testing
  5. Simulation Models as Systems
  6. Simulation Model Validity Domains
  7. Design Contracts
  8. Contract Theory
  9. Component
  10. Variable and Assertion
  11. Contract
  12. Relations and Properties
  13. Satisfaction and Implementation
  14. Refinement
  15. Compatibility and Consistency
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Ontology of components, contracts, variables, and related concepts in the context of simulation models, based on braunSupportingScenarioBasedValidationAutomated2022thesis.
  • Figure 2: Overview of the test case contract, simulation setup contract, and the composition of simulation model contracts.
  • Figure 3: Context for and environment of the framework applying design contracts for simulation models and simulation setups, based on braunSupportingScenarioBasedValidationAutomated2022thesis.