FuzzRisk: Online Collision Risk Estimation for Autonomous Vehicles based on Depth-Aware Object Detection via Fuzzy Inference

TL;DR

The paper addresses the challenge of estimating collision risk for autonomous vehicles when perception is imperfect by comparing a depth-based retrieval of near-field objects with a conventional 3D detector. It uses $IoU$ and depth discrepancy to quantify inconsistencies, and employs a fuzzy inference system optimized against the USC risk metric to convert these signals into a real-time collision-risk score. The approach is validated on the nuScenes dataset and in closed-loop simulations, showing that depth-based cues can recover objects missed by the detector under challenging conditions and can be used to trigger protective actions. The work contributes a practical, interpretable, monocular perception monitor that links perception performance directly to safety risk and offers three FIS construction strategies for robust risk estimation.

Abstract

This paper presents a novel monitoring framework that infers the level of collision risk for autonomous vehicles (AVs) based on their object detection performance. The framework takes two sets of predictions from different algorithms and associates their inconsistencies with the collision risk via fuzzy inference. The first set of predictions is obtained by retrieving safety-critical 2.5D objects from a depth map, and the second set comes from the ordinary AV's 3D object detector. We experimentally validate that, based on Intersection-over-Union (IoU) and a depth discrepancy measure, the inconsistencies between the two sets of predictions strongly correlate to the error of the 3D object detector against ground truths. This correlation allows us to construct a fuzzy inference system and map the inconsistency measures to an AV collision risk indicator. In particular, we optimize the fuzzy inference system towards an existing offline metric that matches AV collision rates well. Lastly, we validate our monitor's capability to produce relevant risk estimates with the large-scale nuScenes dataset and demonstrate that it can safeguard an AV in closed-loop simulations.

Figures (6)

• Figure 1: Our overall monitoring framework uses alignment measures between two sets of predictions to infer ego vehicle's collision risks. The dashed lines mark an offline optimization process using a previously presented risk-correlating metric, USC liao2024usc.
• Figure 2: We implement a sequence of image processing techniques to find the safety-critical objects from a given depth map. The images from left to right depict (a) the given depth map, (b) the inverse of the depth map, (c) foreground removal on the inverse, and (d) the bounding boxes rendered on the original image. The first three images have been normalized to the intensity range $[0, 255]$ for visualization.
• Figure 3: Construction and output surfaces of the FIS via three different approaches, namely (a) knowledge-based crafting, (b) data-driven membership tuning, and (c) data-driven rule learning. For simplicity, the tilde above the variables $\mathsf{RDD}$ and $\mathsf{IoU}$ for denoting frame-wise values are omitted.
• Figure 4: Qualitative results of our monitor applied to the nuScenes dataset caesar2020nuscenes. Blue boxes are from the 3D object detector wang2021probabilistic, and yellow boxes are from the depth-based pipeline. Our monitor successfully translates object detection alignment measures into a useful collision risk indicator.
• Figure 5: In a simulated scenario with an unseen cow (top), our monitor successfully raises the risk level and triggers a controller shield to decelerate the AV (bottom; green). On the contrary, the baseline AV without the monitor and the shield would continue to drive and hit the cow (bottom; blue).