Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAM 3D for 3D Object Reconstruction from Remote Sensing Images

Junsheng Yao, Lichao Mou, Qingyu Li

TL;DR

The paper addresses monocular remote-sensing 3D building reconstruction by evaluating SAM 3D, a general-purpose image-to-3D foundation model, against TRELLIS on NYC data. It introduces a segment--reconstruct--compose pipeline to extend single-object reconstructions to urban scenes and uses perceptual metrics (FID and CMMD) to quantify realism and semantic alignment. Results show SAM 3D yields sharper roof geometry and better boundaries for individual buildings and demonstrates potential for scene-level modeling, while also highlighting practical limitations such as per-building inference cost and lack of inter-object constraints. The work provides actionable guidance for deploying foundation models in urban 3D reconstruction and motivates integrating scene-level priors for improved coherence.

Abstract

Monocular 3D building reconstruction from remote sensing imagery is essential for scalable urban modeling, yet existing methods often require task-specific architectures and intensive supervision. This paper presents the first systematic evaluation of SAM 3D, a general-purpose image-to-3D foundation model, for monocular remote sensing building reconstruction. We benchmark SAM 3D against TRELLIS on samples from the NYC Urban Dataset, employing Frechet Inception Distance (FID) and CLIP-based Maximum Mean Discrepancy (CMMD) as evaluation metrics. Experimental results demonstrate that SAM 3D produces more coherent roof geometry and sharper boundaries compared to TRELLIS. We further extend SAM 3D to urban scene reconstruction through a segment-reconstruct-compose pipeline, demonstrating its potential for urban scene modeling. We also analyze practical limitations and discuss future research directions. These findings provide practical guidance for deploying foundation models in urban 3D reconstruction and motivate future integration of scene-level structural priors.

SAM 3D for 3D Object Reconstruction from Remote Sensing Images

TL;DR

The paper addresses monocular remote-sensing 3D building reconstruction by evaluating SAM 3D, a general-purpose image-to-3D foundation model, against TRELLIS on NYC data. It introduces a segment--reconstruct--compose pipeline to extend single-object reconstructions to urban scenes and uses perceptual metrics (FID and CMMD) to quantify realism and semantic alignment. Results show SAM 3D yields sharper roof geometry and better boundaries for individual buildings and demonstrates potential for scene-level modeling, while also highlighting practical limitations such as per-building inference cost and lack of inter-object constraints. The work provides actionable guidance for deploying foundation models in urban 3D reconstruction and motivates integrating scene-level priors for improved coherence.

Abstract

Monocular 3D building reconstruction from remote sensing imagery is essential for scalable urban modeling, yet existing methods often require task-specific architectures and intensive supervision. This paper presents the first systematic evaluation of SAM 3D, a general-purpose image-to-3D foundation model, for monocular remote sensing building reconstruction. We benchmark SAM 3D against TRELLIS on samples from the NYC Urban Dataset, employing Frechet Inception Distance (FID) and CLIP-based Maximum Mean Discrepancy (CMMD) as evaluation metrics. Experimental results demonstrate that SAM 3D produces more coherent roof geometry and sharper boundaries compared to TRELLIS. We further extend SAM 3D to urban scene reconstruction through a segment-reconstruct-compose pipeline, demonstrating its potential for urban scene modeling. We also analyze practical limitations and discuss future research directions. These findings provide practical guidance for deploying foundation models in urban 3D reconstruction and motivate future integration of scene-level structural priors.
Paper Structure (10 sections, 2 equations, 3 figures, 1 table)

This paper contains 10 sections, 2 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Experiments
  4. Dataset
  5. Experimental setup
  6. Evaluation Metrics
  7. Results
  8. Quantitative Comparison on Individual Buildings
  9. Urban Scene Reconstruction
  10. Conclusion

Figures (3)

  • Figure 1: SAM 3D architecture for evaluation-driven 3D building reconstruction from monocular remote sensing imagery.
  • Figure 2: Visual comparison of reconstructed buildings. Each row corresponds to one building. Columns 1/2 show the input remote-sensing image and the corresponding GT view; columns 3--4 show SAM 3D and TRELLIS results. Columns 5--10 repeat the GT/SAM 3D/TRELLIS triplets for two additional viewpoints.
  • Figure 3: Urban scene reconstruction with SAM 3D. From left to right: remote sensing image, building instance layout, scene reconstruction result, and ground truth. Red boxes indicate buildings with incorrect orientation.