Integrity Monitoring of 3D Object Detection in Automated Driving Systems using Raw Activation Patterns and Spatial Filtering

TL;DR

The paper tackles reliable hazard detection in automated driving beyond collisions by defining triggering events and introducing generalized, event-agnostic metrics that fuse kinematic cues with dynamic safe-distance calculations, such as $d_{safe}^{long}$ and $d_{safe}^{lat}$. It also presents a dataset-agnostic hazardous-event annotation pipeline that computes actor kinematics from basic data via a sliding-window approach and Ego-perspective rotation to apply the metrics. Key contributions include the explicit longitudinal and lateral safe-distance formulas, a modular set of kinematic triggers, and a scalable annotation framework that demonstrates high detection accuracy (~$99.5$–$99.7\%$) on the GTACrash benchmark when combining metrics. The work supports safer ADS perception by enabling early, standardized hazard labeling and highlights the need for diverse real-world near-miss datasets and broader hazard categories to validate robustness across domains.

Abstract

The deep neural network (DNN) models are widely used for object detection in automated driving systems (ADS). Yet, such models are prone to errors which can have serious safety implications. Introspection and self-assessment models that aim to detect such errors are therefore of paramount importance for the safe deployment of ADS. Current research on this topic has focused on techniques to monitor the integrity of the perception mechanism in ADS. Existing introspection models in the literature, however, largely concentrate on detecting perception errors by assigning equal importance to all parts of the input data frame to the perception module. This generic approach overlooks the varying safety significance of different objects within a scene, which obscures the recognition of safety-critical errors, posing challenges in assessing the reliability of perception in specific, crucial instances. Motivated by this shortcoming of state of the art, this paper proposes a novel method integrating raw activation patterns of the underlying DNNs, employed by the perception module, analysis with spatial filtering techniques. This novel approach enhances the accuracy of runtime introspection of the DNN-based 3D object detections by selectively focusing on an area of interest in the data, thereby contributing to the safety and efficacy of ADS perception self-assessment processes.

Figures (2)

• Figure 1: Demonstration of our data-agnostic hazardous event labelling technique, tested on a pre-labelled simulation benchmark, GTACrash. The triggering event was detected using vehicle kinematics and calculated safe distances.
• Figure 2: Dataset annotation flowchart