Unsupervised Learning for Detection of Rare Driving Scenarios

TL;DR

This work tackles the critical problem of detecting rare and hazardous driving scenarios for autonomous systems using unsupervised learning. It introduces a Deep Isolation Forest (DIF) framework applied to Driving Anomaly Data (DAD) derived from multimodal naturalistic driving data, with t-SNE used for visualization. The contributions include applying DIF in this domain, a robust data processing and feature engineering pipeline, and an evaluation strategy leveraging a proxy ground truth and perceptual validation. Findings show that DIF outperforms baselines in identifying non-linear, rare events, offering a scalable approach while acknowledging limitations of proxy labels and feature engineering that guide future improvements.

Abstract

The detection of rare and hazardous driving scenarios is a critical challenge for ensuring the safety and reliability of autonomous systems. This research explores an unsupervised learning framework for detecting rare and extreme driving scenarios using naturalistic driving data (NDD). We leverage the recently proposed Deep Isolation Forest (DIF), an anomaly detection algorithm that combines neural network-based feature representations with Isolation Forests (IFs), to identify non-linear and complex anomalies. Data from perception modules, capturing vehicle dynamics and environmental conditions, is preprocessed into structured statistical features extracted from sliding windows. The framework incorporates t-distributed stochastic neighbor embedding (t-SNE) for dimensionality reduction and visualization, enabling better interpretability of detected anomalies. Evaluation is conducted using a proxy ground truth, combining quantitative metrics with qualitative video frame inspection. Our results demonstrate that the proposed approach effectively identifies rare and hazardous driving scenarios, providing a scalable solution for anomaly detection in autonomous driving systems. Given the study's methodology, it was unavoidable to depend on proxy ground truth and manually defined feature combinations, which do not encompass the full range of real-world driving anomalies or their nuanced contextual dependencies.

Figures (9)

• Figure 1: Illustration of the pipeline for detecting safety-critical driving scenarios, showcasing the integration of ML-based and rule-based detectors to identify anomalies and respond effectively.
• Figure 2: Synthetic data example showing hard anomalies in the original space (left) and their representation in the projected space by DIF (right).
• Figure 3: Overview of the novel anomaly detection framework: The figure depicts the flow of vehicle bus signals and perception signals (derived from perception modules) through processing and feature engineering steps, followed by anomaly detection using Deep Isolation Forest. The model takes Driving Anomaly Data as input, a multivariate tabular dataset where rows represent windows and columns represent features. The output consists of anomaly scores for each window. A threshold is defined to classify windows as anomalies if their anomaly scores exceed the threshold, while those below it are considered normal. Finally, the detection results are evaluated using proxy ground truth set derived from the Driving Anomaly Data.
• Figure 4: Data processing and feature engineering pipeline: Multivariate time-series signals from vehicle and perception modules are segmented into windows, aggregated into statistical features, and preprocessed into a tabular dataset for anomaly detection.
• Figure 5: Anomaly score distributions for events labeled as anomalous by proxy heuristics vs. randomly sampled normal events. Higher scores indicate stronger anomaly signals by DIF