CloudFixer: Test-Time Adaptation for 3D Point Clouds via Diffusion-Guided Geometric Transformation

TL;DR

CloudFixer introduces a diffusion-guided, per-sample input adaptation for 3D point clouds to combat real-world noise and distribution shifts. By optimizing a geometry-based transformation y_phi(x) (rotations and per-point displacements) to align with a diffusion model trained on the source domain, and by avoiding backpropagation through the diffusion model, it achieves real-time adaptation. An optional online model adaptation (CloudFixer-O) further aligns predictions between original and adapted inputs, improving robustness under batch and stream conditions. Extensive experiments on ModelNet40-C and PointDA-10 across multiple backbones show state-of-the-art performance under challenging and mild conditions, with strong efficiency and broad architectural applicability. The approach demonstrates practical impact for on-device 3D perception, offering a principled, diffusion-informed pathway to reliable test-time adaptation in 3D vision.

Abstract

3D point clouds captured from real-world sensors frequently encompass noisy points due to various obstacles, such as occlusion, limited resolution, and variations in scale. These challenges hinder the deployment of pre-trained point cloud recognition models trained on clean point clouds, leading to significant performance degradation. While test-time adaptation (TTA) strategies have shown promising results on this issue in the 2D domain, their application to 3D point clouds remains under-explored. Among TTA methods, an input adaptation approach, which directly converts test instances to the source domain using a pre-trained diffusion model, has been proposed in the 2D domain. Despite its robust TTA performance in practical situations, naively adopting this into the 3D domain may be suboptimal due to the neglect of inherent properties of point clouds, and its prohibitive computational cost. Motivated by these limitations, we propose CloudFixer, a test-time input adaptation method tailored for 3D point clouds, employing a pre-trained diffusion model. Specifically, CloudFixer optimizes geometric transformation parameters with carefully designed objectives that leverage the geometric properties of point clouds. We also substantially improve computational efficiency by avoiding backpropagation through the diffusion model and a prohibitive generation process. Furthermore, we propose an online model adaptation strategy by aligning the original model prediction with that of the adapted input. Extensive experiments showcase the superiority of CloudFixer over various TTA baselines, excelling in handling common corruptions and natural distribution shifts across diverse real-world scenarios. Our code is available at https://github.com/shimazing/CloudFixer

Figures (13)

• Figure 1: Accuracy of TENT wang2020tent and CloudFixer across different batch sizes and label distributions (Random Order vs. Class Order).
• Figure 2: CloudFixer is an optimization-based diffusion-guided input adaptation, tailored for 3D point clouds. CloudFixer iteratively optimizes geometric transformation parameters $\phi$ for $x$ to minimize the Chamfer distance between a parameterized point cloud $y_\phi$ and the estimation $\hat{y}$ from the diffusion model, aligned with the source domain. A distorted input point cloud $x$, leading to a misclassification, is transformed into $y_{\phi*}$, correcting its prediction. Additionally, online model adaptation minimizes the KL-divergence between class predictions of the original and adapted point clouds.
• Figure 3: The average accuracy of CloudFixer and other baselines across all corruptions in ModelNet40-C for various classifier architectures.
• Figure 4: Point cloud visualization examples demonstrate the effects of CloudFixer on various common corruption types in ModelNet40-C and natural distributions in PointDA-10. The upper row showcases corrupted examples, while the lower row illustrates the corresponding results after applying CloudFixer.
• Figure 5: Accuracy on PointDA-10 using DGCNN. We report the performance of all method, except MEMO, DDA, CloudFixer, including CloudFixer-O with a batch size of 64.