Defining Operational Conditions for Safety-Critical AI-Based Systems from Data

TL;DR

This paper addresses how to define safe operational conditions for AI-based safety-critical systems by deriving the Operational Design Domain (ODD) from data using a deterministic kernel-based framework. Anchor points establish local affinities that combine into a global affinity $\alpha$, with membership determined by a threshold $\zeta$ and safeguards for out-of-distribution samples via $\xi$; the method is implemented in the autoSAFE toolkit. Validation via Monte Carlo experiments and a real-world aviation use case (VCAS) demonstrates that the data-driven ODD can closely approximate the true ODD, with the convex hull serving as a tunable proxy for risk thresholds and monitoring. The approach enables continuous, interpretable, and certifiable safety boundaries for data-driven AI systems, while future work explores temporal dynamics, cross-dimensional dependencies, and integration into formal certification workflows.

Abstract

Artificial Intelligence (AI) has been on the rise in many domains, including numerous safety-critical applications. However, for complex systems found in the real world, or when data already exist, defining the underlying environmental conditions is extremely challenging. This often results in an incomplete description of the environment in which the AI-based system must operate. Nevertheless, this description, called the Operational Design Domain (ODD), is required in many domains for the certification of AI-based systems. Traditionally, the ODD is created in the early stages of the development process, drawing on sophisticated expert knowledge and related standards. This paper presents a novel Safety-by-Design method to a posteriori define the ODD from previously collected data using a multi-dimensional kernel-based representation. This approach is validated through both Monte Carlo methods and a real-world aviation use case for a future safety-critical collision-avoidance system. Moreover, by defining under what conditions two ODDs are equal, the paper shows that the data-driven ODD can equal the original, underlying hidden ODD of the data. Utilizing the novel, Safe-by-Design kernel-based ODD enables future certification of data-driven, safety-critical AI-based systems.

Figures (4)

• Figure 1: Setup for the 2D example of the Monte-Carlo method. The blue dots are the anchor points from which the data-driven ODD $\mathcal{O}$ is derived. The red dots are the validation samples to compare the data-driven ODD $\mathcal{O}$ to the underlying original ODD.
• Figure 2: Precision-recall curves for a data-driven ODD where the underlying ODD is defined as $(X = [-5, 5] \times [-5, 5], \mathcal{R} = \{R_1: x_2 \geq x_1-3\})$ (cf. \ref{['fig:2D-MCM']}). The results show that similar precision-recall curves can be derived from the convex hull compared to the original underlying ODD.
• Figure 3: Geometry of the vertical collision avoidance scenario for VCAS, from Julian2019. The black ownship is trying to avoid the (malicious) red intruder by diverting through climbing or descending.
• Figure 4: Resulting precision-recall curves for the VCAS use case. Again, a strong similarity between the underlying original ODD (left) and the convex hull over all anchor points (right) is noticeable, confirming the data-driven ODD approach.

Theorems & Definitions (7)

• Definition 3.1: Equality of ODD Structures
• Definition 4.1: Samples and Anchor Points
• Definition 4.2: Local Affinity Function
• Definition 4.3: Global ODD Affinity Function
• Definition 4.4: Threshold-Based ODD Membership
• Definition 4.5: OOD Consistency Constraint
• Definition 4.6: Kernel-Based Operational Design Domain