Test-Time Adaptation of 3D Point Clouds via Denoising Diffusion Models

TL;DR

This paper introduces a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation, which uses a diffusion strategy that adapts input point cloud samples to the source domain while keeping the source model parameters intact.

Abstract

Test-time adaptation (TTA) of 3D point clouds is crucial for mitigating discrepancies between training and testing samples in real-world scenarios, particularly when handling corrupted point clouds. LiDAR data, for instance, can be affected by sensor failures or environmental factors, causing domain gaps. Adapting models to these distribution shifts online is crucial, as training for every possible variation is impractical. Existing methods often focus on fine-tuning pre-trained models based on self-supervised learning or pseudo-labeling, which can lead to forgetting valuable source domain knowledge over time and reduce generalization on future tests. In this paper, we introduce a novel 3D test-time adaptation method, termed 3DD-TTA, which stands for 3D Denoising Diffusion Test-Time Adaptation. This method uses a diffusion strategy that adapts input point cloud samples to the source domain while keeping the source model parameters intact. The approach uses a Variational Autoencoder (VAE) to encode the corrupted point cloud into a shape latent and latent points. These latent points are corrupted with Gaussian noise and subjected to a denoising diffusion process. During this process, both the shape latent and latent points are updated to preserve fidelity, guiding the denoising toward generating consistent samples that align more closely with the source domain. We conduct extensive experiments on the ShapeNet dataset and investigate its generalizability on ModelNet40 and ScanObjectNN, achieving state-of-the-art results. The code has been released at \url{https://github.com/hamidreza-dastmalchi/3DD-TTA}.

Figures (5)

• Figure 1: Reconstruction of corrupted point clouds using the proposed 3DD-TTA method.
• Figure 2: In the TTA setting, the source model encounters corrupted 3D point clouds with an unknown distribution shift, requiring adaptation without prior knowledge of the corruption type. Our 3DD-TTA approach adds Gaussian noise to the latent points (encoded by a pre-trained VAE) to reduce their dependence on the initial corruption. The distribution of the Chamfer Distance (CD) between original and corrupted point clouds—corrupted with uniform noise, Gaussian noise, and RBF rbf—from ShapeNet dataset shapenet is shown before (Left) and after (Right) Gaussian perturbation. After perturbation, the CD distributions for all corruption types overlap, demonstrating corruption independence.
• Figure 3: Given a corrupted test point cloud $\tilde{\textbf{x}}$, we adapt it to the source domain to improve classifier $p_c$. Using the LION model lion, the point cloud is encoded into shape latent $\textbf{z}_0$ and latent points $\textbf{h}_0$. The latent points $\textbf{h}_0$, perturbed with Gaussian noise, are progressively denoised over $t_w$ iterations. During denoising, both shape latent and latent points are updated to minimize the SCD distance$l^{\lambda}_{cd}$. After denoising, the latent points are decoded into the adapted point cloud and passed to the classifier $p_c$ for prediction.
• Figure 4: Qualitative assessment of the proposed test-time adaptation across various corruptions.
• Figure 5: (left) Accuracy of the source classifier after adaptation using different numbers of denoising steps. (right) Performance of the model across different numbers of total DDIM steps.