A Machine Learning Perspective on Automated Driving Corner Cases

TL;DR

The paper tackles safety-critical corner cases in autonomous driving by redefining CCs as departures from the training data distribution, formalized as $P_{D_O}(\\tilde{x}_O,\\tilde{y}) \\approx 0$. It proposes a data-driven, two-branch framework that detects semantic CCs via open-world segmentation with uncertainty-based models and covariate CCs via global OOD detection in latent space or pixel-wise uncertainty aggregates. Key contributions include a formal data-driven CC definition, a unified detection framework, and new datasets/benchmarks (e.g., Foggy Lost & Found) to study combined CCs. The approach demonstrates strong performance across benchmarks and offers a scalable, principled path toward reliable corner-case recognition in real-world autonomous driving systems.

Abstract

For high-stakes applications, like autonomous driving, a safe operation is necessary to prevent harm, accidents, and failures. Traditionally, difficult scenarios have been categorized into corner cases and addressed individually. However, this example-based categorization is not scalable and lacks a data coverage perspective, neglecting the generalization to training data of machine learning models. In our work, we propose a novel machine learning approach that takes the underlying data distribution into account. Based on our novel perspective, we present a framework for effective corner case recognition for perception on individual samples. In our evaluation, we show that our approach (i) unifies existing scenario-based corner case taxonomies under a distributional perspective, (ii) achieves strong performance on corner case detection tasks across standard benchmarks for which we extend established out-of-distribution detection benchmarks, and (iii) enables analysis of combined corner cases via a newly introduced fog-augmented Lost & Found dataset. These results provide a principled basis for corner case recognition, underlining our manual specification-free definition.

Figures (5)

• Figure 1: Illustration of Corner Cases in relation to the empirical training distribution. Training samples are marked with black circles, and Corner Cases are marked with red circles. We distinguish between the semantic Corner Case and the co-variate Corner Case, which are low-density regions in the training distribution, but have different data properties.
• Figure 2: Data Driven Corner Case Framework: We aim to detect the CCs in autonomous driving, which have been previously defined primarily on exemplar scenarios, based on membership of the training distribution. Therefore, we define a semantic and a co-variate Corner Case. For semantic we employ open-world segmentation and extend it by a sample-level OOD detection to detect co-variate CCs.
• Figure 3: Evaluation of semantic corner case: Detection of novel objects (frisbee and bear) of U3HS and P2F on COCO Lin2014. It can be observed that both open-world segmentation approaches consistently detect instances of novel objects. Images are from Schmidt2025b
• Figure 4: Visual Anomaly Segmentation performance of U3HS and P2F on L&F and Foggy L&F.
• Figure 5: TSNE-Plot of OOD Datasets considering CS as InD.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3