Relative Energy Learning for LiDAR Out-of-Distribution Detection

TL;DR

This work tackles LiDAR OOD detection in open-world autonomous driving by introducing Relative Energy Learning (REL), which uses a relative energy margin $\Delta E(x;f) = \log \frac{\sum_{i\in y^-} \exp(f_i(x))}{\sum_{i\in y^+} \exp(f_i(x))}$ to separate in-distribution and OOD evidence, paired with a lightweight Point Raise method to synthesize realistic pseudo-OOD samples. Implemented as a small OOD projector on the Mask4Former-3D backbone, REL preserves strong ID segmentation while producing dense per-point OOD scores. Empirically, REL achieves state-of-the-art OOD detection on STU and SemanticKITTI, reducing FPR@95 significantly and improving AUROC and AP, with ablations showing the gains stem from both Point Raise and the REL objective. The approach is simple, scalable, and effective for open-world LiDAR perception, offering a principled path toward safer autonomous driving in the presence of unknown objects, including unseen unlabeled content. The method crucially relies on the relative energy margin and a single imbalance weight, making it robust across scenes without extensive hyperparameter tuning.

Abstract

Out-of-distribution (OOD) detection is a critical requirement for reliable autonomous driving, where safety depends on recognizing road obstacles and unexpected objects beyond the training distribution. Despite extensive research on OOD detection in 2D images, direct transfer to 3D LiDAR point clouds has been proven ineffective. Current LiDAR OOD methods struggle to distinguish rare anomalies from common classes, leading to high false-positive rates and overconfident errors in safety-critical settings. We propose Relative Energy Learning (REL), a simple yet effective framework for OOD detection in LiDAR point clouds. REL leverages the energy gap between positive (in-distribution) and negative logits as a relative scoring function, mitigating calibration issues in raw energy values and improving robustness across various scenes. To address the absence of OOD samples during training, we propose a lightweight data synthesis strategy called Point Raise, which perturbs existing point clouds to generate auxiliary anomalies without altering the inlier semantics. Evaluated on SemanticKITTI and the Spotting the Unexpected (STU) benchmark, REL consistently outperforms existing methods by a large margin. Our results highlight that modeling relative energy, combined with simple synthetic outliers, provides a principled and scalable solution for reliable OOD detection in open-world autonomous driving.

Figures (6)

• Figure 1: Point-level OOD detection performance on the STU test set. Each method is positioned by its AUROC (↑) and FPR@95 (↓). Our method achieves state-of-the-art performance, with the highest AUROC and lowest FPR@95.
• Figure 2: Overview of the Relative Energy Learning (REL) framework. We adopt the Mask4Former yilmaz2024mask4former backbone with a sparse UNet encoder choy2019minkowski and a transformer decoder operating on Farthest Point Sampled (FPS) queries qi2017pointnetdeephierarchicalfeatureschult2023mask3d. The transformer decoder produces closed-set segmentation masks, while in parallel, a lightweight OOD branch generates dense point-wise energy scores. The OOD branch is trained with the REL objective, and its outputs are converted into uncertainty maps for OOD detection.
• Figure 3: Visualization of Raise Cluster with Different $\gamma$.
• Figure 4: Visualization of anomaly segmentation results on the STU benchmark. Points are categorized into three types: inlier, anomaly, and unlabeled. The unlabeled class covers a wide range of objects that may appear in driving scenes but are excluded from training and evaluation, such as rubbish bins. These objects are OOD relative to the closed-set training data, yet they are unlikely to pose a threat to the vehicle. We use a continuous gradient (color bar) to visualize each point’s predicted likelihood of belonging to ID or OOD. Our method produces high-quality anomaly segmentation while keeping the false positive rate on inliers low.
• Figure 5: Visualization of anomaly segmentation results on the STU benchmark. Points are categorized into three types: inlier, anomaly, and unlabeled. For visualization, predictions are shown with a continuous gradient from ID to OOD.