Implicit Guidance and Explicit Representation of Semantic Information in Points Cloud: A Survey
Jingyuan Tang, Yuhuan Zhao, Songlin Sun, Yangang Cai
TL;DR
The paper surveys the use of semantic information in 3D point clouds, articulating two core roles: implicit guidance that improves traditional tasks (compression, registration, reconstruction, etc.) and explicit representation that yields direct semantic outputs (segmentation, 3D dense captioning, scene graphs). It systematically reviews datasets and methods across traditional and emerging tasks, highlighting how semantic cues from 2D vision inform 3D processing and how 3D semantics are advancing tasks like scene understanding and language- grounded reasoning. Key contributions include a taxonomy aligning tasks under implicit and explicit semantics, a dataset-centric benchmarking perspective, and forward-looking discussions on challenges such as data scarcity, cross-modal fusion, and the need for better SSC metrics. The study also points to pragmatic directions, including semi-/unsupervised learning, cross-modal datasets, and NeRF-based approaches to enhance semantic-aware 3D understanding with real-world impact in autonomous systems, robotics, and immersive environments.
Abstract
Point clouds, a prominent method of 3D representation, are extensively utilized across industries such as autonomous driving, surveying, electricity, architecture, and gaming, and have been rigorously investigated for their accuracy and resilience. The extraction of semantic information from scenes enhances both human understanding and machine perception. By integrating semantic information from two-dimensional scenes with three-dimensional point clouds, researchers aim to improve the precision and efficiency of various tasks. This paper provides a comprehensive review of the diverse applications and recent advancements in the integration of semantic information within point clouds. We explore the dual roles of semantic information in point clouds, encompassing both implicit guidance and explicit representation, across traditional and emerging tasks. Additionally, we offer a comparative analysis of publicly available datasets tailored to specific tasks and present notable observations. In conclusion, we discuss several challenges and potential issues that may arise in the future when fully utilizing semantic information in point clouds, providing our perspectives on these obstacles. The classified and organized articles related to semantic based point cloud tasks, and continuously followed up on relevant achievements in different fields, which can be accessed through https://github.com/Jasmine-tjy/Semantic-based-Point-Cloud-Tasks.