A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions

TL;DR

This survey surveys the task of text-guided 3D visual grounding (T-3DVG), detailing its problem scope, common pipeline components, and the spectrum of methods from two-stage detectors to end-to-end one-stage regressors, including weakly supervised and large-model approaches. It catalogs foundational datasets (ScanRefer, Nr3D, Sr3D), evaluation metrics, and performance trends, while analyzing the strengths and limitations of each architectural category. The authors discuss multi-modal data fusion, contextual encoders, and cross-modal reasoning strategies, and they highlight the growing role of 2D extra views, RGB-D inputs, and large language models in improving grounding accuracy. Finally, the paper identifies key challenges—annotation burden, background clutter, and practicality—and outlines future directions such as zero-shot learning, dense paragraph-level grounding, and better feature extractors to advance real-world applicability. The work serves as a comprehensive roadmap for researchers and practitioners aiming to develop robust, efficient, and scalable T-3DVG systems.

Abstract

Text-guided 3D visual grounding (T-3DVG), which aims to locate a specific object that semantically corresponds to a language query from a complicated 3D scene, has drawn increasing attention in the 3D research community over the past few years. Compared to 2D visual grounding, this task presents great potential and challenges due to its closer proximity to the real world and the complexity of data collection and 3D point cloud source processing. In this survey, we attempt to provide a comprehensive overview of the T-3DVG progress, including its fundamental elements, recent research advances, and future research directions. To the best of our knowledge, this is the first systematic survey on the T-3DVG task. Specifically, we first provide a general structure of the T-3DVG pipeline with detailed components in a tutorial style, presenting a complete background overview. Then, we summarize the existing T-3DVG approaches into different categories and analyze their strengths and weaknesses. We also present the benchmark datasets and evaluation metrics to assess their performances. Finally, we discuss the potential limitations of existing T-3DVG and share some insights on several promising research directions. The latest papers are continually collected at https://github.com/liudaizong/Awesome-3D-Visual-Grounding.

Figures (17)

• Figure 1: An illustration of the text-guided 3D visual grounding (T-3DVG).
• Figure 2: Statistics of the collected papers in this survey. Left: number of papers published in each year. Right: distribution of papers by venue.
• Figure 3: A general pipeline for the T-3DVG task, consisting of multi-modal feature extractors, multi-modal feature encoders, multi-modal interaction module, and the final grounding head. An additional object detector is further introduced for two-stage T-3DVG methods to pre-extract all possible object proposals in each scene.
• Figure 4: Illustration of two types of scene context encoders. They learn more dependencies among intra-modal instances (objects or points).
• Figure 5: Illustration of two types of multi-modal interaction methods.