Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Taxonomy of Real-World Defeaters in Safety Assurance Cases

Usman Gohar, Michael C. Hunter, Myra B. Cohen, Robyn R. Lutz

TL;DR

Safety-critical CPS assurance cases are vulnerable to defeaters that undermine claim validity. The authors conduct a systematic literature review and thematic analysis to derive a seven-category defeater taxonomy, grounded in real-world cases. They validate the taxonomy with an initial application to a sUAS case and release open artifacts to support adoption. The framework aims to standardize defeater analysis, improving robustness and regulatory confidence, with future work on empirical validation and tool support.

Abstract

The rise of cyber-physical systems in safety-critical domains calls for robust risk-evaluation frameworks. Assurance cases, often required by regulatory bodies, are a structured approach to demonstrate that a system meets its safety requirements. However, assurance cases are fraught with challenges, such as incomplete evidence and gaps in reasoning, called defeaters, that can call into question the credibility and robustness of assurance cases. Identifying these defeaters increases confidence in the assurance case and can prevent catastrophic failures. The search for defeaters in an assurance case, however, is not structured, and there is a need to standardize defeater analysis. The software engineering community thus could benefit from having a reusable classification of real-world defeaters in software assurance cases. In this paper, we conducted a systematic study of literature from the past 20 years. Using open coding, we derived a taxonomy with seven broad categories, laying the groundwork for standardizing the analysis and management of defeaters in safety-critical systems. We provide our artifacts as open source for the community to use and build upon, thus establishing a common framework for understanding defeaters.

A Taxonomy of Real-World Defeaters in Safety Assurance Cases

TL;DR

Safety-critical CPS assurance cases are vulnerable to defeaters that undermine claim validity. The authors conduct a systematic literature review and thematic analysis to derive a seven-category defeater taxonomy, grounded in real-world cases. They validate the taxonomy with an initial application to a sUAS case and release open artifacts to support adoption. The framework aims to standardize defeater analysis, improving robustness and regulatory confidence, with future work on empirical validation and tool support.

Abstract

The rise of cyber-physical systems in safety-critical domains calls for robust risk-evaluation frameworks. Assurance cases, often required by regulatory bodies, are a structured approach to demonstrate that a system meets its safety requirements. However, assurance cases are fraught with challenges, such as incomplete evidence and gaps in reasoning, called defeaters, that can call into question the credibility and robustness of assurance cases. Identifying these defeaters increases confidence in the assurance case and can prevent catastrophic failures. The search for defeaters in an assurance case, however, is not structured, and there is a need to standardize defeater analysis. The software engineering community thus could benefit from having a reusable classification of real-world defeaters in software assurance cases. In this paper, we conducted a systematic study of literature from the past 20 years. Using open coding, we derived a taxonomy with seven broad categories, laying the groundwork for standardizing the analysis and management of defeaters in safety-critical systems. We provide our artifacts as open source for the community to use and build upon, thus establishing a common framework for understanding defeaters.

Paper Structure

This paper contains 19 sections, 3 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Motivation and Background
  3. Current practice
  4. Overview of the Automated Flight Authorization System
  5. Survey Methodology
  6. Identifying relevant studies
  7. Analyzing data and collating results
  8. Threats to Validity
  9. Taxonomy
  10. Logical Fallacies
  11. Contextual Defeaters
  12. Evidence Validity Defeaters
  13. Requirements Engineering Defeaters
  14. Structural Defeaters
  15. Adversarial Defeaters
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Overview of the safety assurance case lifecycle, illustrating the roles of various stakeholders.
  • Figure 2: sUAS Safety Assurance Case fragment with example defeaters.
  • Figure 3: A high-level taxonomy of real-world defeaters in assurance cases. The boxes at the top show the seven broad categories.