UOD: Unseen Object Detection in 3D Point Cloud

TL;DR

This work tackles open-world 3D object detection in Lidar point clouds by proposing a unified evaluation protocol for unseen object localization and OOD classification, and by introducing four practical methods: anomaly sample augmentation, universal objectness, detecting unseen objects, and distinguishing unseen objects. It combines anomaly-based data augmentation with an auxiliary objectness head and energy-based, outlier-aware learning to improve both detection and OOD discrimination across multiple mainstream detectors. The authors validate their approach on KITTI Misc and two synthetic OOD benchmarks (Nuscenes OOD and SUN-RGBD OOD), showing consistent, significant performance gains over strong baselines and a recent SOTA method. The results offer a practical pathway toward robust, safe 3D perception in open-world autonomous driving settings, with broad applicability to real-world perception systems.

Abstract

Existing 3D object detectors encounter extreme challenges in localizing unseen 3D objects and recognizing them as unseen, which is a crucial technology in autonomous driving in the wild. To address these challenges, we propose practical methods to enhance the performance of 3D detection and Out-Of-Distribution (OOD) classification for unseen objects. The proposed methods include anomaly sample augmentation, learning of universal objectness, learning of detecting unseen objects, and learning of distinguishing unseen objects. To demonstrate the effectiveness of our approach, we propose the KITTI Misc benchmark and two additional synthetic OOD benchmarks: the Nuscenes OOD benchmark and the SUN-RGBD OOD benchmark. The proposed methods consistently enhance performance by a large margin across all existing methods, giving insight for future work on unseen 3D object detection in the wild.

Figures (9)

• Figure 1: Base 3D object detector and our method comparison. (a): 3D object detection result of baseline SECOND yan2018second on KITTI geiger2012we 'Misc' class object; (b): Comparison of the base detector and our method in two aspects: unseen object detection performance (Recall) and OOD classification performance (AUROC).
• Figure 2: In-Distribution (ID)(seen) and Out-Of-Distribution (OOD)(unseen) object localization performance comparison. This plot illustrates the recall for both ID (seen) and OOD (unseen) objects based on the proposal number. This depicts the recall for OOD objects at IoU thresholds of 0.1, 0.25, and 0.4.
• Figure 3: Visualization result on SUN-RGBD song2015sun pointcloud of original and resized object. (a): Point cloud of the original object for Anomaly Sample Augmentation; (b): Point cloud of the resized object for Multi-size Mix Augmentation.
• Figure 4: Visualization on our proposed synthetic benchmark. The blue box represents the original ID (seen) object, while the green box represents our cut-pasted synthesized OOD (unseen) object.
• Figure 5: OOD (unseen) object recall comparison on KITTI Misc benchmark. OOD(B) represents the result of the baseline detector SECOND, and OOD(P) represents the result of our method on SECOND.