Towards Long-Range 3D Object Detection for Autonomous Vehicles

TL;DR

This work tackles long-range 3D object detection for autonomous vehicles by addressing sparsity-induced domain gaps and label imbalance beyond $100\,\text{m}$. It proposes two complementary strategies: range experts (two distance-specialized detectors with distance-aware training and late fusion) and Multimodal Virtual Points (MVP) from image-based depth completion to densify LiDAR data. Experiments on the AV2 dataset show MVP yields strong long-range improvements and that range experts offer a computationally efficient alternative without heavy camera dependencies; combining them via late fusion provides the best overall gains. The findings highlight the potential of distance-aware training and image-guided depth augmentation to enhance safe, reliable perceptual range in autonomous driving systems, with practical implications for highway and heavy-vehicle scenarios.

Abstract

3D object detection at long range is crucial for ensuring the safety and efficiency of self driving vehicles, allowing them to accurately perceive and react to objects, obstacles, and potential hazards from a distance. But most current state of the art LiDAR based methods are range limited due to sparsity at long range, which generates a form of domain gap between points closer to and farther away from the ego vehicle. Another related problem is the label imbalance for faraway objects, which inhibits the performance of Deep Neural Networks at long range. To address the above limitations, we investigate two ways to improve long range performance of current LiDAR based 3D detectors. First, we combine two 3D detection networks, referred to as range experts, one specializing at near to mid range objects, and one at long range 3D detection. To train a detector at long range under a scarce label regime, we further weigh the loss according to the labelled point's distance from ego vehicle. Second, we augment LiDAR scans with virtual points generated using Multimodal Virtual Points (MVP), a readily available image-based depth completion algorithm. Our experiments on the long range Argoverse2 (AV2) dataset indicate that MVP is more effective in improving long range performance, while maintaining a straightforward implementation. On the other hand, the range experts offer a computationally efficient and simpler alternative, avoiding dependency on image-based segmentation networks and perfect camera-LiDAR calibration.

Figures (2)

• Figure 1: Number of labeled objects for all categories as a function of range for the training set of the dataset. The first two bins, which we refer to as mid-range, contain 85.8% of the events in the data set. The 100-150 m range contains most of the long range objects, 10.9%, leaving only 3.2% objects over 150 m.
• Figure 2: Increased sparsity with distance from ego vehicle is one of the major challenges facing long range (>100 m) 3D object detection. (a) shows that the two vehicles parked far away from the ego vehicle have fewer points, as well as appear different compared to those parked nearby. To address this problem, we analyze two approaches: (b) adding virtual points during training, and (c) training two neural networks, one specializing in mid-range object detection, and one for detecting long-range objects. The parameters $r_1,r_2 \in [0,250]$, represented by the green and blue box respectively, delimit a subset of points used for training the long and mid-ranged experts respectively. Mid-range expert uses points within a distance $r_2$ of the ego vehicle, while points outside the distance $r_1$ are used to train the long-range expert. (d) During inference, we combine outputs from the two networks, such that the long-range detector contributes detections beyond blue box (>100 m), and the mid-range detector contributes detections within blue box.