Constructing Safety Cases for AI Systems: A Reusable Template Framework

TL;DR

The paper addresses the inadequacy of traditional safety cases for frontier AI by developing a reusable, AI-specific CAE framework with taxonomies for claims, arguments, and evidence. It introduces a library of templates and four end-to-end patterns (discovery-driven evaluation, marginal-risk without ground truth, continuous evolution, and threshold-based acceptability) and demonstrates how these can be composed within a dynamic assurance pipeline (Builder, Validator, Registry). A case study on an AI-based tender evaluation system illustrates how marginal-risk reasoning and comparative evidence can justify deployment without ground-truth, while maintaining auditability and governance. The approach offers a scalable, auditable foundation for ongoing AI safety assurance amid continual model updates and context shifts, and highlights open challenges in scalability, interoperability, and evaluation reproducibility. Overall, the framework aims to operationalize safety cases for frontier AI through templates, patterns, and live governance artifacts that evolve with the system.

Abstract

Safety cases, structured arguments that a system is acceptably safe, are becoming central to the governance of AI systems. Yet, traditional safety-case practices from aviation or nuclear engineering rely on well-specified system boundaries, stable architectures, and known failure modes. Modern AI systems such as generative and agentic AI are the opposite. Their capabilities emerge unpredictably from low-level training objectives, their behaviour varies with prompts, and their risk profiles shift through fine-tuning, scaffolding, or deployment context. This study examines how safety cases are currently constructed for AI systems and why classical approaches fail to capture these dynamics. It then proposes a framework of reusable safety-case templates, each following a predefined structure of claims, arguments, and evidence tailored for AI systems. The framework introduces comprehensive taxonomies for AI-specific claim types (assertion-based, constrained-based, capability-based), argument types (demonstrative, comparative, causal/explanatory, risk-based, and normative), and evidence families (empirical, mechanistic, comparative, expert-driven, formal methods, operational/field data, and model-based). Each template is illustrated through end-to-end patterns addressing distinctive challenges such as evaluation without ground truth, dynamic model updates, and threshold-based risk decisions. The result is a systematic, composable, and reusable approach to constructing and maintaining safety cases that are credible, auditable, and adaptive to the evolving behaviour of generative and frontier AI systems.

Figures (8)

• Figure 1: Claims, Arguments, Evidence (CAE) example
• Figure 2: AI safety case template, CAE taxonomy, and pattern
• Figure 3: Overview of AI safety cases ecosystem pipeline
• Figure 4: Architectural block diagram for this research paper
• Figure 5: Primary study selection process steps