An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains

TL;DR

This work provides an empirical, architecture-focused study of how LiDAR-based 3D object detectors generalize to unseen domains, isolating the effects of four design choices—architecture, voxel encoding, data augmentations, and anchor strategies—across six benchmarks spanning sensor, location, and weather gaps. By benchmarking nine detectors with apples-to-apples evaluations and conducting extensive controlled experiments, the authors identify practical levers for robustness, notably that transformer backbones with local point features improve OOD performance, test-time anchor tuning yields large cross-domain gains, and source-domain augmentations help with low-resolution sensors, while clean-weather training enhances weather robustness. The findings culminate in actionable guidance for building more robust 3D-OD systems prior to full-domain adaptation, and they highlight that some robustness challenges (e.g., bad weather) may be discriminability-driven rather than transferability-driven. These insights enable safer deployment by informing detector design choices, data curation, and test-time adaptation strategies in real-world autonomous systems.

Abstract

3D Object Detectors (3D-OD) are crucial for understanding the environment in many robotic tasks, especially autonomous driving. Including 3D information via Lidar sensors improves accuracy greatly. However, such detectors perform poorly on domains they were not trained on, i.e. different locations, sensors, weather, etc., limiting their reliability in safety-critical applications. There exist methods to adapt 3D-ODs to these domains; however, these methods treat 3D-ODs as a black box, neglecting underlying architectural decisions and source-domain training strategies. Instead, we dive deep into the details of 3D-ODs, focusing our efforts on fundamental factors that influence robustness prior to domain adaptation. We systematically investigate four design choices (and the interplay between them) often overlooked in 3D-OD robustness and domain adaptation: architecture, voxel encoding, data augmentations, and anchor strategies. We assess their impact on the robustness of nine state-of-the-art 3D-ODs across six benchmarks encompassing three types of domain gaps - sensor type, weather, and location. Our main findings are: (1) transformer backbones with local point features are more robust than 3D CNNs, (2) test-time anchor size adjustment is crucial for adaptation across geographical locations, significantly boosting scores without retraining, (3) source-domain augmentations allow the model to generalize to low-resolution sensors, and (4) surprisingly, robustness to bad weather is improved when training directly on more clean weather data than on training with bad weather data. We outline our main conclusions and findings to provide practical guidance on developing more robust 3D-ODs.

Figures (8)

• Figure 1: An overview of the three main domain gaps under study. We show the main shift types for each domain gap along with the underlying design choices that influence model robustness.
• Figure 2: Evaluation of different anchor sizes (in the order of increasing volumes) at test-time on W$\rightarrow$K car benchmark. Three training experiments using SECOND are performed, each with a different anchor size. Using the same size at test-time results in a poor performance ($\circ$), but a very high peak can be obtained by going for lower anchor sizes.
• Figure 3: Evaluation of all models on K64, K16, K64$\rightarrow$K16 and K16$\rightarrow$K64 benchmarks. The mAP for all classes is reported. The performance drops significantly from high-to-low resolution while it increases from low-to-high at test-time.
• Figure 4: We train each model of (VOTR-TSD, PVRCNN, and SECOND) on K64 with three different voxel heights. We evaluate these nine training experiments on K64, and the lower resolutions K32, and K16. mAP for all classes is reported. Larger heights increase performance on the unseen target domains (2-7 mAP points), while keeping the source performance relatively stable (2-3 mAP points).
• Figure 5: Left: Source domain with 64 lines. Right: In a target domain with 32 lines, a large number of voxels will be empty if the voxel height is too small (blue voxel), but this number will be reduced if the voxel height is increased (red voxel).