Sparse3DPR: Training-Free 3D Hierarchical Scene Parsing and Task-Adaptive Subgraph Reasoning from Sparse RGB Views

TL;DR

Sparse3DPR addresses the accuracy-efficiency gap in training-free 3D scene understanding from sparse RGB views by building a hierarchical plane-enhanced scene graph (HPSG) anchored by dominant planes and enriched with open-vocabulary semantics. It introduces a task-adaptive subgraph extraction that prunes irrelevant context, enabling efficient and accurate reasoning with LLMs. The framework achieves significant improvements on Space3D-Bench (EM@1 +28.7% and 78.2% speedup over ConceptGraphs) and competitive results on ScanQA against training-based methods, with real-world robustness demonstrated. By combining geometry-driven scene parsing with open-vocabulary semantics and adaptive reasoning, Sparse3DPR offers a practical, generalizable solution for open-ended 3D scene understanding.

Abstract

Recently, large language models (LLMs) have been explored widely for 3D scene understanding. Among them, training-free approaches are gaining attention for their flexibility and generalization over training-based methods. However, they typically struggle with accuracy and efficiency in practical deployment. To address the problems, we propose Sparse3DPR, a novel training-free framework for open-ended scene understanding, which leverages the reasoning capabilities of pre-trained LLMs and requires only sparse-view RGB inputs. Specifically, we introduce a hierarchical plane-enhanced scene graph that supports open vocabulary and adopts dominant planar structures as spatial anchors, which enables clearer reasoning chains and more reliable high-level inferences. Furthermore, we design a task-adaptive subgraph extraction method to filter query-irrelevant information dynamically, reducing contextual noise and improving 3D scene reasoning efficiency and accuracy. Experimental results demonstrate the superiority of Sparse3DPR, which achieves a 28.7% EM@1 improvement and a 78.2% speedup compared with ConceptGraphs on the Space3D-Bench. Moreover, Sparse3DPR obtains comparable performance to training-based methods on ScanQA, with additional real-world experiments confirming its robustness and generalization capability.

Figures (3)

• Figure 1: Comparison of 3D scene understanding methods. Unlike existing methods requiring dense 3D inputs or training, Sparse3DPR leverages sparse RGB views to construct a hierarchical plane-enhanced scene graph (HPSG) and performs task-adaptive subgraph extraction for efficient LLM‑based scene understanding.
• Figure 2: Overview of Sparse3DPR framework. Sparse3DPR is a training-free framework for 3D scene understanding from sparse RGB views. It parses the scene through two branches that integrate 3D geometry: one extracts structural elements by applying class-agnostic masks combined with structural plane detection and labeling, while the other uses open-vocabulary masks for semantic-geometric fusion to obtain object instances and generate captions. These components form a candidate pair pool of topological connections, which are further refined by an LLM to estimate spatial relations between object pairs and construct the HPSG. A task-adaptive subgraph extractor then selects relevant context from the HPSG for LLM-based reasoning.
• Figure 3: Qualitative results. We showcase Sparse3DPR performing object, spatial, and geometric understanding in a lab scene and locating target objects within the scene. These examples demonstrate its ability to adapt to diverse tasks and generalize to complex real-world indoor scenes.