Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking LiDAR Domain Generalization: Single Source as Multiple Density Domains

Jaeyeul Kim, Jungwan Woo, Jeonghoon Kim, Sunghoon Im

TL;DR

The paper tackles domain generalization for LiDAR semantic segmentation across diverse sensors by exploiting intra-source density variations. It introduces Density Discriminative Feature Embedding (DDFE), a modular framework comprising beam density estimation, density soft clipping, and density-aware embedding, complemented by a density augmentation strategy to broaden the source density spectrum. Empirical results across Waymo, nuScenes, and SemanticKITTI show that DDFE improves cross-domain generalization over state-of-the-art DA/DG methods with lightweight integration into 3D backbones. This approach offers a practical, scalable path to robust LiDAR perception in unseen environments without target-domain data or ego-motion reliance, and code is publicly available.

Abstract

In the realm of LiDAR-based perception, significant strides have been made, yet domain generalization remains a substantial challenge. The performance often deteriorates when models are applied to unfamiliar datasets with different LiDAR sensors or deployed in new environments, primarily due to variations in point cloud density distributions. To tackle this challenge, we propose a Density Discriminative Feature Embedding (DDFE) module, capitalizing on the observation that a single source LiDAR point cloud encompasses a spectrum of densities. The DDFE module is meticulously designed to extract density-specific features within a single source domain, facilitating the recognition of objects sharing similar density characteristics across different LiDAR sensors. In addition, we introduce a simple yet effective density augmentation technique aimed at expanding the spectrum of density in source data, thereby enhancing the capabilities of the DDFE. Our DDFE stands out as a versatile and lightweight domain generalization module. It can be seamlessly integrated into various 3D backbone networks, where it has demonstrated superior performance over current state-of-the-art domain generalization methods. Code is available at https://github.com/dgist-cvlab/MultiDensityDG.

Rethinking LiDAR Domain Generalization: Single Source as Multiple Density Domains

TL;DR

The paper tackles domain generalization for LiDAR semantic segmentation across diverse sensors by exploiting intra-source density variations. It introduces Density Discriminative Feature Embedding (DDFE), a modular framework comprising beam density estimation, density soft clipping, and density-aware embedding, complemented by a density augmentation strategy to broaden the source density spectrum. Empirical results across Waymo, nuScenes, and SemanticKITTI show that DDFE improves cross-domain generalization over state-of-the-art DA/DG methods with lightweight integration into 3D backbones. This approach offers a practical, scalable path to robust LiDAR perception in unseen environments without target-domain data or ego-motion reliance, and code is publicly available.

Abstract

In the realm of LiDAR-based perception, significant strides have been made, yet domain generalization remains a substantial challenge. The performance often deteriorates when models are applied to unfamiliar datasets with different LiDAR sensors or deployed in new environments, primarily due to variations in point cloud density distributions. To tackle this challenge, we propose a Density Discriminative Feature Embedding (DDFE) module, capitalizing on the observation that a single source LiDAR point cloud encompasses a spectrum of densities. The DDFE module is meticulously designed to extract density-specific features within a single source domain, facilitating the recognition of objects sharing similar density characteristics across different LiDAR sensors. In addition, we introduce a simple yet effective density augmentation technique aimed at expanding the spectrum of density in source data, thereby enhancing the capabilities of the DDFE. Our DDFE stands out as a versatile and lightweight domain generalization module. It can be seamlessly integrated into various 3D backbone networks, where it has demonstrated superior performance over current state-of-the-art domain generalization methods. Code is available at https://github.com/dgist-cvlab/MultiDensityDG.
Paper Structure (23 sections, 9 equations, 4 figures, 5 tables)

This paper contains 23 sections, 9 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LiDAR-based Semantic Segmentation
  4. LiDAR-based Domain Adaptation
  5. LiDAR-based Domain Generalization
  6. Method
  7. Point-voxel feature encoding
  8. Beam density estimation module
  9. Density soft clipping
  10. Density-aware embedding module
  11. Density Augmentation
  12. Experiments
  13. Implementation Details
  14. Datasets
  15. Comparison to State-of-the-Art DA/DG Methods
  16. ...and 8 more sections

Figures (4)

  • Figure 1: Motivation of our DDFE: Leveraging Diverse Densities in a Source LiDAR for Domain Generalization - Despite the apparent differences in density distributions between Waymo (64-channel), SemanticKITTI (64-channel), and nuScenes (32-channel), they share regions of overlapping density distributions. For example, the observation that a vehicle at 35 meters in Waymo (top) has a similar density to one at 25 meters in SemanticKITTI (middle) and 12 meters in nuScenes (bottom) underscores this phenomenon. This understanding of varying local density within a source domain serves as a foundation of our domain generalization method. The proposed DDFE transforms features from 3D space to a unified density space without additional training on unseen data, enhancing domain generalization performance.
  • Figure 2: Overview of the DDFE pipeline.
  • Figure 3: Qualitative comparison with MinkNet42 backbone. Top: Model trained on nuScenes, tested on SemanticKITTI (N$\rightarrow$K). Bottom: Trained on SemanticKITTI, tested on nuScenes (K$\rightarrow$N).
  • Figure 4: Visualization of feature similarity matrices between the Waymo (64-channel) and the nuScenes (32-channel) datasets, with a focus on the distance of objects from the LiDAR. We utilize models trained with the nuScenes dataset for visualization. (a) A point-voxel feature encoding method. (b) A density-aware embedding module. (c) A density soft clipping. Our DDFE combines all modules (a), (b), and (c).