Point Cloud Understanding via Attention-Driven Contrastive Learning

TL;DR

This work introduces PointACL, an attention-driven contrastive learning framework designed to address limitations of Transformer-based models, which employs an attention-driven dynamic masking strategy that guides the model to focus on under-attended regions, enhancing the understanding of global structures within the point cloud.

Abstract

Recently Transformer-based models have advanced point cloud understanding by leveraging self-attention mechanisms, however, these methods often overlook latent information in less prominent regions, leading to increased sensitivity to perturbations and limited global comprehension. To solve this issue, we introduce PointACL, an attention-driven contrastive learning framework designed to address these limitations. Our method employs an attention-driven dynamic masking strategy that guides the model to focus on under-attended regions, enhancing the understanding of global structures within the point cloud. Then we combine the original pre-training loss with a contrastive learning loss, improving feature discrimination and generalization. Extensive experiments validate the effectiveness of PointACL, as it achieves state-of-the-art performance across a variety of 3D understanding tasks, including object classification, part segmentation, and few-shot learning. Specifically, when integrated with different Transformer backbones like Point-MAE and PointGPT, PointACL demonstrates improved performance on datasets such as ScanObjectNN, ModelNet40, and ShapeNetPart. This highlights its superior capability in capturing both global and local features, as well as its enhanced robustness against perturbations and incomplete data.

Figures (6)

• Figure 1: Illustration of PointACL's Advantages. Point-MAE is employed as the backbone of our proposed PointACL. Left: PointACL emphasizes extracting global information from a greater number of patches. Right: PointACL demonstrates greater robustness than previous methods.
• Figure 2: Overview of the PointACL Framework. PointACL consists of two branches that share the same weights: a standard mode branch and a masked mode branch. An attention-driven dynamic masking module generates a masked point cloud by selecting less activated patches from the output of the standard mode branch. Both branches process their respective inputs through the shared Transformer blocks to obtain latent representations. Finally, a joint contrastive loss is used to align the representations of these two branches.
• Figure 3: Gaussian noise analysis on ScanObjectNN. While the performance of existing methods decmidrules sharply with Gaussian noise, this issue is mitigated by incorporating PointACL.
• Figure 4: Attention visualization of PointACL with Point-MAE and PointGPT. Patches with high attention are closer to red, while patches with low attention are closer to blue. Point-MAE is employed as the backbone of our proposed PointACL.
• Figure 5: Gaussian noise analysis on ScanObjectNN. While the performance of existing methods decmidrules sharply with increasing Gaussian noise, this issue is mitigated by incorporating PointACL. Notably, when Point-MAE is used as the backbone network, our PointACL significantly enhances its robustness, resulting in minimal accuracy degradation.