A Survey of Label-Efficient Deep Learning for 3D Point Clouds

TL;DR

This survey addresses the challenge of learning from limited annotations in 3D point clouds by proposing a taxonomy of label-efficient strategies and surveying four core approaches: data augmentation, domain transfer, weakly-supervised learning, and pretrained foundation models. It provides a comprehensive literature review across tasks (classification, detection, segmentation) and data prerequisites, discusses benchmarking outcomes, and highlights current challenges and future directions. The work emphasizes practical impact, including reduced annotation costs, better generalization across domains, and the potential of multimodal and self-supervised pretraining to bootstrap 3D perception. Overall, it maps a rapidly evolving field, identifying key methods, trade-offs, and opportunities for open research and real-world deployment.

Abstract

In the past decade, deep neural networks have achieved significant progress in point cloud learning. However, collecting large-scale precisely-annotated training data is extremely laborious and expensive, which hinders the scalability of existing point cloud datasets and poses a bottleneck for efficient exploration of point cloud data in various tasks and applications. Label-efficient learning offers a promising solution by enabling effective deep network training with much-reduced annotation efforts. This paper presents the first comprehensive survey of label-efficient learning of point clouds. We address three critical questions in this emerging research field: i) the importance and urgency of label-efficient learning in point cloud processing, ii) the subfields it encompasses, and iii) the progress achieved in this area. To achieve this, we propose a taxonomy that organizes label-efficient learning methods based on the data prerequisites provided by different types of labels. We categorize four typical label-efficient learning approaches that significantly reduce point cloud annotation efforts: data augmentation, domain transfer learning, weakly-supervised learning, and pretrained foundation models. For each approach, we outline the problem setup and provide an extensive literature review that showcases relevant progress and challenges. Finally, we share insights into current research challenges and potential future directions. A project associated with this survey has been built at https://github.com/xiaoaoran/3D_label_efficient_learning.

Figures (15)

• Figure 1: An overview of label-efficient learning for 3D point clouds. With the task of semantic segmentation, we compare traditional fully supervised learning, which demands costly point-wise annotations, with label-efficient learning strategies prioritizing minimal annotation efforts. These strategies encompass data augmentation, weakly supervised learning, domain transfer learning, self-supervised pre-training and multi-modal foundation models. Best viewed in color.
• Figure 2: Taxonomy of label-efficient learning of point clouds.
• Figure 3: Data augmentation in 3D network training.
• Figure 4: Illustration of widely-used conventional augmentation techniques for point clouds.
• Figure 5: Illustration of typical mixing DA methods in point cloud classification, including PointMixup chen2020pointmixup, RSMix lee2021regularization, and SageMix lee2022sagemix.