USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Driving

TL;DR

This work presents uncompromising spatial constraints (USC), which characterize a simple yet important localization requirement demanding the predictions to fully cover the objects when seen from the autonomous vehicle, and incorporates the quantitative measures into common loss functions to enable safety-oriented fine-tuning for existing models.

Abstract

In this work, we consider the safety-oriented performance of 3D object detectors in autonomous driving contexts. Specifically, despite impressive results shown by the mass literature, developers often find it hard to ensure the safe deployment of these learning-based perception models. Attributing the challenge to the lack of safety-oriented metrics, we hereby present uncompromising spatial constraints (USC), which characterize a simple yet important localization requirement demanding the predictions to fully cover the objects when seen from the autonomous vehicle. The constraints, as we formulate using the perspective and bird's-eye views, can be naturally reflected by quantitative measures, such that having an object detector with a higher score implies a lower risk of collision. Finally, beyond model evaluation, we incorporate the quantitative measures into common loss functions to enable safety-oriented fine-tuning for existing models. With experiments using the nuScenes dataset and a closed-loop simulation, our work demonstrates such considerations of safety notions at the perception level not only improve model performances beyond accuracy but also allow for a more direct linkage to actual system safety.

Figures (7)

• Figure 1: An example scene that motivates our safety-oriented spatial constraints. For both the blue and the red predictions, $\mathsf{IoU}\approx0.7$ and $\mathsf{TE}\approx1.6 \, (m)$ indistinguishably. We thereby focus on the objects' nearest sides from the AV and propose USC. Effectively, the blue prediction has $\mathsf{USC}=1$, while the red one gets $\mathsf{USC}\approx0.8$.
• Figure 2: We transform 3D bounding boxes (BBs) onto the PV and BEV planes for establishing USC and corresponding quantitative measures. Gold: original BB; Blue: converted PV or BEV BB.
• Figure 3: A comparison of IoU and IoGT using the same ground truth (gray) and different predictions (blue) in (a) and (b). The prediction in (b) should be indicated as better than the one in (a) when considering collision avoidance. However, the IoU in (b) is lower, falling short to reflect the notion of "enclosure".
• Figure 4: An example showing the risk where with the prediction's closest point $v_\mathbf{P}^c$ placed closer than the ground truth's $v_\mathbf{G}^c$, parts of the ground truth may still be exposed. Hence, we suggest taking into account the entire AV-facing sides, which are defined by the right-most and left-most corners from the AV's perspective.
• Figure 5: Range-based model evaluation results. The upper plot shows the scores of the four examined models for objects ranging in $[0, 10]$ meters, and the lower $[10, 20]$ meters. The camera-based FCOS3D wang2021fcos3d and PGD wang2021probabilistic are plotted in green, and the lidar-based SSN zhu2020ssn and CenterPoint yin2021center are in blue.