Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STEAM & MoSAFE: SOTIF Error-and-Failure Model & Analysis for AI-Enabled Driving Automation

Krzysztof Czarnecki, Hiroshi Kuwajima

TL;DR

This work advances safety assurance for AI-enabled driving by introducing STEAM, a temporal refinement of the SOTIF cause-and-effect model, and MoSAFE, a model-based analysis framework that instantiates STEAM from system-design models. STEAM adds hazardous error sequences at the element level and hazardous behavior patterns at the vehicle level, enabling temporally structured causal reasoning across perception, prediction, planning, and control. MoSAFE operationalizes this by two activities: hazard/severity evaluation using high- and detailed-level models, and functional-insufficiency/triggering-condition analysis with weakest-precondition reasoning to map hazards down to component inputs. A braking/ACC case study demonstrates how MoSAFE derives hazard patterns, evaluates their severity and likelihood, and supports safety-case evidence, while simulations validate key insights and quantify residual risk. The approach offers a modular, assume-guarantee framework that complements simulation-based testing and supports safety cases for complex DAS architectures, including AI components, by systematically linking hazard patterns to acceptance criteria and safety requirements.

Abstract

Driving Automation Systems (DAS) are subject to complex road environments and vehicle behaviors and increasingly rely on sophisticated sensors and Artificial Intelligence (AI). These properties give rise to unique safety faults stemming from specification insufficiencies and technological performance limitations, where sensors and AI introduce errors that vary in magnitude and temporal patterns, posing potential safety risks. The Safety of the Intended Functionality (SOTIF) standard emerges as a promising framework for addressing these concerns, focusing on scenario-based analysis to identify hazardous behaviors and their causes. Although the current standard provides a basic cause-and-effect model and high-level process guidance, it lacks concepts required to identify and evaluate hazardous errors, especially within the context of AI. This paper introduces two key contributions to bridge this gap. First, it defines the SOTIF Temporal Error and Failure Model (STEAM) as a refinement of the SOTIF cause-and-effect model, offering a comprehensive system-design perspective. STEAM refines error definitions, introduces error sequences, and classifies them as error sequence patterns, providing particular relevance to systems employing advanced sensors and AI. Second, this paper proposes the Model-based SOTIF Analysis of Failures and Errors (MoSAFE) method, which allows instantiating STEAM based on system-design models by deriving hazardous error sequence patterns at module level from hazardous behaviors at vehicle level via weakest precondition reasoning. Finally, the paper presents a case study centered on an automated speed-control feature, illustrating the practical applicability of the refined model and the MoSAFE method in addressing complex safety challenges in DAS.

STEAM & MoSAFE: SOTIF Error-and-Failure Model & Analysis for AI-Enabled Driving Automation

TL;DR

This work advances safety assurance for AI-enabled driving by introducing STEAM, a temporal refinement of the SOTIF cause-and-effect model, and MoSAFE, a model-based analysis framework that instantiates STEAM from system-design models. STEAM adds hazardous error sequences at the element level and hazardous behavior patterns at the vehicle level, enabling temporally structured causal reasoning across perception, prediction, planning, and control. MoSAFE operationalizes this by two activities: hazard/severity evaluation using high- and detailed-level models, and functional-insufficiency/triggering-condition analysis with weakest-precondition reasoning to map hazards down to component inputs. A braking/ACC case study demonstrates how MoSAFE derives hazard patterns, evaluates their severity and likelihood, and supports safety-case evidence, while simulations validate key insights and quantify residual risk. The approach offers a modular, assume-guarantee framework that complements simulation-based testing and supports safety cases for complex DAS architectures, including AI components, by systematically linking hazard patterns to acceptance criteria and safety requirements.

Abstract

Driving Automation Systems (DAS) are subject to complex road environments and vehicle behaviors and increasingly rely on sophisticated sensors and Artificial Intelligence (AI). These properties give rise to unique safety faults stemming from specification insufficiencies and technological performance limitations, where sensors and AI introduce errors that vary in magnitude and temporal patterns, posing potential safety risks. The Safety of the Intended Functionality (SOTIF) standard emerges as a promising framework for addressing these concerns, focusing on scenario-based analysis to identify hazardous behaviors and their causes. Although the current standard provides a basic cause-and-effect model and high-level process guidance, it lacks concepts required to identify and evaluate hazardous errors, especially within the context of AI. This paper introduces two key contributions to bridge this gap. First, it defines the SOTIF Temporal Error and Failure Model (STEAM) as a refinement of the SOTIF cause-and-effect model, offering a comprehensive system-design perspective. STEAM refines error definitions, introduces error sequences, and classifies them as error sequence patterns, providing particular relevance to systems employing advanced sensors and AI. Second, this paper proposes the Model-based SOTIF Analysis of Failures and Errors (MoSAFE) method, which allows instantiating STEAM based on system-design models by deriving hazardous error sequence patterns at module level from hazardous behaviors at vehicle level via weakest precondition reasoning. Finally, the paper presents a case study centered on an automated speed-control feature, illustrating the practical applicability of the refined model and the MoSAFE method in addressing complex safety challenges in DAS.
Paper Structure (24 sections, 21 equations, 13 figures, 5 tables)

This paper contains 24 sections, 21 equations, 13 figures, 5 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Background: SOTIF Cause-and-Effect Model and Assurance Process
  4. SOTIF Temporal Error and Failure Model (STEAM)
  5. Model-based SOTIF Analysis of Failures and Errors (MoSAFE)
  6. Identification and Evaluation of Hazards (Clause 6)
  7. Hazard Identification
  8. Hazard Evaluation and Acceptance Criteria Specification
  9. Model-based Severity Evaluation
  10. Identification and Evaluation of Functional Insufficiencies & Triggering Conditions (Clause 7)
  11. Model-based Identification and Evaluation of Functional Insufficiencies at the Element Level
  12. Identification and Likelihood Estimation for -based Components
  13. Additional Considerations and Future Work
  14. Designing Adequate Environment and System Models
  15. Misbehavior Injection
  16. ...and 9 more sections

Figures (13)

  • Figure 1: SOTIF cause-and-effect model (based on Figs. 3B and 4 in ISO21448), including new elements of SOTIF Temporal Error and Failure Model (STEAM) in red
  • Figure 2: High-level scenario model of the RVE, capturing the HBSC, and the driving policy for "braking for a stationary vehicle ahead", with the injection logic for marked in red
  • Figure 3: The (left) experiences an unintended braking interruption () when braking for a POVPOV (right) stopped ahead. The transforms a safe situation (top) into an unsafe one (bottom).
  • Figure 4: Sample braking speed profiles for $v_\text{init}=15\,m/s$ and $s_\text{stop}=112.5\,m$
  • Figure 5: Illustration of the patterns from Tab. \ref{['tab:UBI']} as sets of sequences. Each glyph in the Venn diagram represents a particular sequence $\rho_a$ and its color indicates the severity of a crash that it would cause in the analyzed scenario. Note that the hazardous patterns are tight over-approximations of their corresponding severity range; e.g., $\mathbb{P}_{a,\text{S3}}$ contains all S3 sequences, but it also contains sequences of all other severities. The patterns are tight given their simple form as in eq. \ref{['eq:UBI-pattern']}.
  • ...and 8 more figures