Adaptive Augmentation-Aware Latent Learning for Robust LiDAR Semantic Segmentation

TL;DR

A3Point is proposed, an adaptive augmentation-aware latent learning framework that effectively utilizes a diverse range of augmentations while mitigating the semantic shift, which refers to the change in the semantic meaning caused by augmentations.

Abstract

Adverse weather conditions significantly degrade the performance of LiDAR point cloud semantic segmentation networks by introducing large distribution shifts. Existing augmentation-based methods attempt to enhance robustness by simulating weather interference during training. However, they struggle to fully exploit the potential of augmentations due to the trade-off between minor and aggressive augmentations. To address this, we propose A3Point, an adaptive augmentation-aware latent learning framework that effectively utilizes a diverse range of augmentations while mitigating the semantic shift, which refers to the change in the semantic meaning caused by augmentations. A3Point consists of two key components: semantic confusion prior (SCP) latent learning, which captures the model's inherent semantic confusion information, and semantic shift region (SSR) localization, which decouples semantic confusion and semantic shift, enabling adaptive optimization strategies for different disturbance levels. Extensive experiments on multiple standard generalized LiDAR segmentation benchmarks under adverse weather demonstrate the effectiveness of our method, setting new state-of-the-art results.

Figures (11)

• Figure 1: (a) Point cloud distortions caused by adverse weather conditions. (b) Augmentation at different levels (light, moderate, and heavy), where we adjust the drop ratio (DR) for point drop and jitter std (JS) for random jittering. (c) Visualization of segmentation accuracy under different augmentation levels, showing that aggressive distortions lead to significant performance degradation.
• Figure 2: Demonstration of semantic confusion and semantic shift. (a) Confusion matrices from source (normal weather) and target (adverse weather) domains. Despite domain shifts, semantic confusion remains consistent. (b) Aggressive augmentations alter point cloud density and shape, leading to semantic misalignment (e.g., car$\rightarrow$veg.).
• Figure 3: Pipeline of A3Point. We explore an abundant augmentation space (Sec.\ref{['3.3.2']}) and propose two key components: SCP latent learning to capture inherent semantic confusion (Sec.\ref{['3.3.3']}) and SSR localization to decouple semantic shift (Sec.\ref{['3.3.4']}).
• Figure 3: Different architectures & benchmarks.
• Figure 4: Qualitative results on $[A]\rightarrow[C]$. Significant improvements are marked with boxes.