The Necessity of a Holistic Safety Evaluation Framework for AI-Based Automation Features

TL;DR

The paper argues that AI components labeled as QM can still generate SOTIF hazards in safety-critical driving systems, challenging the assumption that QM implies safety neutrality. It proposes a holistic safety framework that fuses FuSa (ISO 26262), SOTIF (ISO 21448), and AI-specific guidance (ISO/PAS 8800), supported by STPA and CTA hazard analyses, to identify and mitigate risks in AI perception modules. Through case studies focused on Low-Level Perception (LLP) in ADAS and Hands-Off Driving (HOD), it demonstrates how QM-rated AI components can produce hazards with $S>0$ or $C>0$, and how to derive AI safety requirements that satisfy SOTIF acceptance criteria. The work argues for updating safety practices to ensure comprehensive end-to-end safety across all component classifications in modern autonomous driving systems.

Abstract

The intersection of Safety of Intended Functionality (SOTIF) and Functional Safety (FuSa) analysis of driving automation features has traditionally excluded Quality Management (QM) components from rigorous safety impact evaluations. While QM components are not typically classified as safety-relevant, recent developments in artificial intelligence (AI) integration reveal that such components can contribute to SOTIF-related hazardous risks. Compliance with emerging AI safety standards, such as ISO/PAS 8800, necessitates re-evaluating safety considerations for these components. This paper examines the necessity of conducting holistic safety analysis and risk assessment on AI components, emphasizing their potential to introduce hazards with the capacity to violate risk acceptance criteria when deployed in safety-critical driving systems, particularly in perception algorithms. Using case studies, we demonstrate how deficiencies in AI-driven perception systems can emerge even in QM-classified components, leading to unintended functional behaviors with critical safety implications. By bridging theoretical analysis with practical examples, this paper argues for the adoption of comprehensive FuSa, SOTIF, and AI standards-driven methodologies to identify and mitigate risks in AI components. The findings demonstrate the importance of revising existing safety frameworks to address the evolving challenges posed by AI, ensuring comprehensive safety assurance across all component classifications spanning multiple safety standards.

Figures (6)

• Figure 1: SOTIF Risk Evaluation according to ISO 21448 standard ISO21448
• Figure 2: Graphical Description of Low and High level perception role in ADAS application
• Figure 3: Caust Tree Analysis for Collision with oncoming traffic or surrounding
• Figure 4: Determining when to apply ISO/PAS 8800.
• Figure 5: AI system design and V$\&$V phase of the AI safety lifecycle introduced by ISO/PAS 8800.