APCoTTA: Continual Test-Time Adaptation for Semantic Segmentation of Airborne LiDAR Point Clouds

TL;DR

APCoTTA tackles continual test-time adaptation for semantic segmentation of airborne LiDAR point clouds by preserving source knowledge while adapting to evolving unlabeled targets. It introduces three components—dynamic trainable layer selection, entropy-based consistency loss, and randomized parameter interpolation—to mitigate catastrophic forgetting and error accumulation during online adaptation. The authors also establish two corruption-robustness benchmarks, ISPRSC and H3DC, and demonstrate that APCoTTA yields substantial mIoU gains over direct inference and several CTTA baselines on these datasets. This work advances practical deployment of ALS segmentation under changing conditions and provides a principled CTTA framework for 3D point clouds.

Abstract

Airborne laser scanning (ALS) point cloud segmentation is a fundamental task for large-scale 3D scene understanding. In real-world applications, models are typically fixed after training. However, domain shifts caused by changes in the environment, sensor types, or sensor degradation often lead to a decline in model performance. Continuous Test-Time Adaptation (CTTA) offers a solution by adapting a source-pretrained model to evolving, unlabeled target domains. Despite its potential, research on ALS point clouds remains limited, facing challenges such as the absence of standardized datasets and the risk of catastrophic forgetting and error accumulation during prolonged adaptation. To tackle these challenges, we propose APCoTTA, the first CTTA method tailored for ALS point cloud semantic segmentation. We propose a dynamic trainable layer selection module. This module utilizes gradient information to select low-confidence layers for training, and the remaining layers are kept frozen, mitigating catastrophic forgetting. To further reduce error accumulation, we propose an entropy-based consistency loss. By losing such samples based on entropy, we apply consistency loss only to the reliable samples, enhancing model stability. In addition, we propose a random parameter interpolation mechanism, which randomly blends parameters from the selected trainable layers with those of the source model. This approach helps balance target adaptation and source knowledge retention, further alleviating forgetting. Finally, we construct two benchmarks, ISPRSC and H3DC, to address the lack of CTTA benchmarks for ALS point cloud segmentation. Experimental results demonstrate that APCoTTA achieves the best performance on two benchmarks, with mIoU improvements of approximately 9% and 14% over direct inference. The new benchmarks and code are available at https://github.com/Gaoyuan2/APCoTTA.

Figures (12)

• Figure 1: The framework of our proposed method. At time step $t$, the current target domain batch $X_{t}$ is processed with both strong and weak augmentations and fed into the model $f_{\theta_{t}}$. First, the Dynamic Selection of Trainable Layers module identifies low-confidence layers as trainable while freezing the rest to mitigate catastrophic forgetting. Next, the Entropy-Based Consistency Loss module updates the trainable layers using high-confidence samples to reduce error accumulation. Finally, the Randomized Parameter Interpolation module randomly selects a subset of trainable parameters and blends them with their pretrained counterparts to further alleviate forgetting and enhance model stability.
• Figure 2: The LiDAR intensity map of training (a) and testing (b) sets of ISPRS dataset.
• Figure 3: The RGB color map of the training and validation sets of the H3D dataset. The validation set is indicated by the yellow box.
• Figure 4: Visualization of typical corruption types with the largest corruption severity level 5 in our benchmark ISPRSC dataset.
• Figure 5: Visualization of typical corruption types with the largest corruption severity level 5 in our benchmark H3DC dataset.