PointMAC: Meta-Learned Adaptation for Robust Test-Time Point Cloud Completion

TL;DR

PointMAC tackles static encoder rigidity in point-cloud completion by introducing meta-learned test-time adaptation with Bi-Aux Units that generate self-supervised signals. It employs a MAML-inspired inner/outer loop to align auxiliary adaptation with the primary completion objective and introduces Adaptive $\lambda$-Calibration to balance gradient contributions from auxiliary tasks. At inference, per-sample refinement of the shared encoder yields sample-specific completions without additional supervision, achieving state-of-the-art results on ShapeNet, PCN, MVP, and KITTI. This approach demonstrates strong generalization across synthetic and real-world domains, offering a practical path toward robust 3D perception in robotics and augmented reality.

Abstract

Point cloud completion is essential for robust 3D perception in safety-critical applications such as robotics and augmented reality. However, existing models perform static inference and rely heavily on inductive biases learned during training, limiting their ability to adapt to novel structural patterns and sensor-induced distortions at test time. To address this limitation, we propose PointMAC, a meta-learned framework for robust test-time adaptation in point cloud completion. It enables sample-specific refinement without requiring additional supervision. Our method optimizes the completion model under two self-supervised auxiliary objectives that simulate structural and sensor-level incompleteness. A meta-auxiliary learning strategy based on Model-Agnostic Meta-Learning (MAML) ensures that adaptation driven by auxiliary objectives is consistently aligned with the primary completion task. During inference, we adapt the shared encoder on-the-fly by optimizing auxiliary losses, with the decoder kept fixed. To further stabilize adaptation, we introduce Adaptive $λ$-Calibration, a meta-learned mechanism for balancing gradients between primary and auxiliary objectives. Extensive experiments on synthetic, simulated, and real-world datasets demonstrate that PointMAC achieves state-of-the-art results by refining each sample individually to produce high-quality completions. To the best of our knowledge, this is the first work to apply meta-auxiliary test-time adaptation to point cloud completion.

Figures (8)

• Figure 1: Existing point cloud completion models operate with fixed inductive biases at inference, often focusing on structurally stable regions (e.g., the fuselage). When such regions are missing, static inference hinders reasoning over other parts (e.g., the tail), resulting in generic completions. Instead, our model applies test-time adaptation using self-supervised signals, enabling dynamic attention to visible cues and producing sample-specific completions that better match the ground truth.
• Figure 2: Overview of our test-time adaptation method. PointMAC formulates point cloud completion as the primary task and introduces Bi-Aux Units to provide self-supervised signals for test-time adaptation. The encoder $\mathcal{E}^{\mathrm{sh}}$ is shared between primary and auxiliary branches. In the meta-learned test-time training phase (a), sample-specific parameters are updated in the inner adaptation using auxiliary losses, while shared parameters are optimized in the outer alignment via the primary completion loss. In the sample-specific TTA phase (b), adaptation proceeds in three steps: (i) the meta-learned model produces initial completions; (ii) the shared encoder $\mathcal{E}^{\mathrm{sh}}$ is updated via self-supervised losses from Bi-Aux Units; (iii) sample-specific completions are generated, adapted to the unique structure and noise of each input.
• Figure 3: Overview of the proposed Bi-Aux Units, consisting of two self-supervised tasks—Stochastic Masked Reconstruction ($Aux^{\text{smr}}$) and Artifact Denoising ($Aux^{\text{ad}}$). Both branches share the encoder $\mathcal{E}^\text{sh}$ and Token Synergy Integrator $\mathcal{I}_{\text{TSI}}$ to ensure consistent feature conditioning, and output features ($\mathcal{F}^M$, $\mathcal{R}^\mathcal{G}$) that are projected to compute the auxiliary losses $\mathcal{L}_{\text{aux}}^{\text{smr}}$ and $\mathcal{L}_{\text{aux}}^{\text{ad}}$.
• Figure 4: Visualization results on the PCN and ShapeNet datasets. Our method preserves fine-grained structures such as the complex geometry of boats and lamps, plane propellers and tails, and chair back slats, demonstrating strong completion quality and generalization across diverse categories.
• Figure 5: Visualization of the ablation study on different components of our framework.