OWL: Unsupervised 3D Object Detection by Occupancy Guided Warm-up and Large Model Priors Reasoning

TL;DR

This work tackles the challenge of unsupervised 3D object detection by addressing unstable initialization and noisy pseudo-labels. It introduces OWL, a framework comprising Occupancy Guided Warm-Up, Instance-Cued Reasoning with large-model priors, and Weight-adapted Self-training to stabilize learning and refine pseudo-labels. Extensive experiments on KITTI and Waymo Open Dataset show substantial improvements over prior unsupervised methods and competitive performance with some weakly/fully supervised approaches, highlighting the practical potential for annotation-free 3D perception. The approach reduces annotation costs while delivering robust 3D and BEV detection, with broader implications for scalable autonomous driving systems.

Abstract

Unsupervised 3D object detection leverages heuristic algorithms to discover potential objects, offering a promising route to reduce annotation costs in autonomous driving. Existing approaches mainly generate pseudo labels and refine them through self-training iterations. However, these pseudo-labels are often incorrect at the beginning of training, resulting in misleading the optimization process. Moreover, effectively filtering and refining them remains a critical challenge. In this paper, we propose OWL for unsupervised 3D object detection by occupancy guided warm-up and large-model priors reasoning. OWL first employs an Occupancy Guided Warm-up (OGW) strategy to initialize the backbone weight with spatial perception capabilities, mitigating the interference of incorrect pseudo-labels on network convergence. Furthermore, OWL introduces an Instance-Cued Reasoning (ICR) module that leverages the prior knowledge of large models to assess pseudo-label quality, enabling precise filtering and refinement. Finally, we design a Weight-adapted Self-training (WAS) strategy to dynamically re-weight pseudo-labels, improving the performance through self-training. Extensive experiments on Waymo Open Dataset (WOD) and KITTI demonstrate that OWL outperforms state-of-the-art unsupervised methods by over 15.0% mAP, revealing the effectiveness of our method.

Figures (5)

• Figure 1: (a) Illustration of the effects of different backbone initialization strategies. (b) Illustration of how to refine pseudo-labels with reasoning based on instance cues.
• Figure 2: (a) Common errors frequently occur during the process of pseudo-label iteration. (b) The L1 3D AP results of OWL on the WOD test set. (c) The impact of OGW on the performance of unsupervised methods.
• Figure 3: OWL framework. (a) Occupancy Guided Warm-Up (OGW) warms up the 3D backbone, guiding the network to learn spatial and semantic context features of the scene through the self-supervised Occupancy proxy task. (b) Instance-Cued Reasoning (ICR) performs reasoning and judgment on various instant cues based on the knowledge of LLMs, and then filters and refines the pseudo-labels. (c) Weight Adapted Self-training (WAS) adaptively reweights each pseudo-label’s loss, letting the model down-weight low-confidence samples.
• Figure 4: The IoU distribution between pseudo-labels and ground truth is presented.
• Figure 5: Visualization comparison of different methods.