Point-SRA: Self-Representation Alignment for 3D Representation Learning

TL;DR

Point-SRA addresses limitations of fixed masking in 3D MAE by revealing and exploiting masking-ratio complementarity and reconstruction uncertainty. It introduces a dual Self-Representation Alignment (MAE-SRA and MFT-SRA) and a MeanFlow-based probabilistic reconstruction (MFT) to model geometry and semantics jointly, augmented by cross-modal conditioning. A Flow-Conditioned Fine-Tuning architecture transfers distributional knowledge from pre-training to downstream tasks. Empirically, Point-SRA delivers strong improvements across object classification, segmentation, and 3D detection benchmarks, validating the benefits of combining ratio-based representation fusion with trajectory-based distribution learning.

Abstract

Masked autoencoders (MAE) have become a dominant paradigm in 3D representation learning, setting new performance benchmarks across various downstream tasks. Existing methods with fixed mask ratio neglect multi-level representational correlations and intrinsic geometric structures, while relying on point-wise reconstruction assumptions that conflict with the diversity of point cloud. To address these issues, we propose a 3D representation learning method, termed Point-SRA, which aligns representations through self-distillation and probabilistic modeling. Specifically, we assign different masking ratios to the MAE to capture complementary geometric and semantic information, while the MeanFlow Transformer (MFT) leverages cross-modal conditional embeddings to enable diverse probabilistic reconstruction. Our analysis further reveals that representations at different time steps in MFT also exhibit complementarity. Therefore, a Dual Self-Representation Alignment mechanism is proposed at both the MAE and MFT levels. Finally, we design a Flow-Conditioned Fine-Tuning Architecture to fully exploit the point cloud distribution learned via MeanFlow. Point-SRA outperforms Point-MAE by 5.37% on ScanObjectNN. On intracranial aneurysm segmentation, it reaches 96.07% mean IoU for arteries and 86.87% for aneurysms. For 3D object detection, Point-SRA achieves 47.3% AP@50, surpassing MaskPoint by 5.12%.

Figures (7)

• Figure 1: Complementarity under varying mask ratios. (a) Mutual information drops and semantic compression rises with increasing mask ratio. (b) 30% mask (teacher) preserves fine-grained details; 75% mask (student) yields better semantic abstraction.
• Figure 2: Under high masking ratios, diverse chair reconstructions from the same masked input (identical visible regions) reflect inherent uncertainty in geometric generation and plausible variations in leg shape, backrest angle, and seat thickness.
• Figure 3: Overview of the Point-SRA. The input point is partitioned via FPS and KNN. Image and text inputs are encoded by pre-trained Vision/Text Transformers. A dual self-representation alignment mechanism adopts a teacher-student MAE. The teacher uses a 30% masking ratio to retain geometry, and the student uses 75% to learn semantics. The teacher is updated via EMA. The MFT reconstructs probabilistic distributions and aligns representations across time. Sg denotes gradient stop.
• Figure 4: Flow-conditioned Fine-Tuning Architecture of Point-SRA. The pretrained MFT (with frozen parameters) takes the center coordinates and outputs flow vectors sampled at different time steps.
• Figure 5: MFT and teacher-student hyperparameter analysis.