Revisiting Out-of-Distribution Detection in LiDAR-based 3D Object Detection

TL;DR

The paper tackles the challenge of detecting out-of-distribution (OOD) objects in LiDAR-based 3D object detection by introducing a lightweight post-hoc OOD detector that operates on a fixed base detector. It synthesizes OOD examples through axis-wise scaling of ID objects and trains a simple MLP to distinguish ID from OOD using fused features from the detector’s backbone and outputs. A novel evaluation protocol tailored for object-level OOD in autonomous driving is proposed, and the nuScenes OOD benchmark is introduced to provide a realistic testing ground. Empirical results show the method outperforms existing baselines on key OOD metrics, with ablations validating the design choices around feature fusion, target maps, and OOD synthesis. The approach offers a practical, architecture-agnostic solution with minimal overhead for improving safety-critical OOD recognition in real-world LiDAR systems.

Abstract

LiDAR-based 3D object detection has become an essential part of automated driving due to its ability to localize and classify objects precisely in 3D. However, object detectors face a critical challenge when dealing with unknown foreground objects, particularly those that were not present in their original training data. These out-of-distribution (OOD) objects can lead to misclassifications, posing a significant risk to the safety and reliability of automated vehicles. Currently, LiDAR-based OOD object detection has not been well studied. We address this problem by generating synthetic training data for OOD objects by perturbing known object categories. Our idea is that these synthetic OOD objects produce different responses in the feature map of an object detector compared to in-distribution (ID) objects. We then extract features using a pre-trained and fixed object detector and train a simple multilayer perceptron (MLP) to classify each detection as either ID or OOD. In addition, we propose a new evaluation protocol that allows the use of existing datasets without modifying the point cloud, ensuring a more authentic evaluation of real-world scenarios. The effectiveness of our method is validated through experiments on the newly proposed nuScenes OOD benchmark. The source code is available at https://github.com/uulm-mrm/mmood3d.

Figures (4)

• Figure 1: A scene containing a car, a pedestrian, and two unknown objects. ID objects are visualized in green, and OOD objects are visualized in red. Our proposed method is able to correctly classify the two unknown objects as OOD while still correctly classifying the known classes as ID. Camera image for visualization purposes only.
• Figure 2: Overview of our proposed framework. We synthesize OOD objects by randomly selecting ID objects and scaling them by an unusual amount. This allows for the training of an MLP in a supervised manner. First, features $F_\text{feat}$ are extracted from the feature map $F$ of the object detector using the bounding boxes $B$. Then, the bounding boxes $B$ are encoded into $F_\text{box}$, the logits $L$ and the one-hot encoded classes $\hat{Y}$ are concatenated and encoded into $F_\text{cls}$. Finally, these features are concatenated with $F_\text{feat}$ to get $F_\text{all}$, which is used as input for the MLP. During training, the ground truth annotations are used instead of the predictions to obtain $B$, $L$, and $\hat{Y}$.
• Figure 3: Example of the proposed OOD object synthesis. (a) two ID objects (truck), (b) the resulting OOD objects
• Figure 4: Qualitative results of the OOD object detection on the nuScenes validation set. ID objects are visualized in green and OOD objects in red. The OOD classification threshold is chosen such that the true positive rate is $95\,\%$. Camera image for visualization purposes only.