Urban Neural Surface Reconstruction from Constrained Sparse Aerial Imagery with 3D SAR Fusion

TL;DR

The paper tackles geometric ambiguity in neural surface reconstruction (NSR) for urban scenes under sparse-view aerial imagery by fusing 3D SAR point clouds with optical imagery. It introduces an SDF-based NSR backbone supervised by multi-modal data, incorporating radar-derived on-surface constraints and radar-guided structure-aware ray sampling and adaptive ray bounding to improve stability and efficiency. A first urban NSR dataset, SARMV3D, is constructed with co-registered aerial imagery and 3D SAR points to benchmark cross-modal urban reconstruction. Experiments show substantial improvements in accuracy, completeness, and robustness over single-modality baselines, especially under oblique and highly constrained views, demonstrating a viable path toward scalable high-fidelity urban reconstruction using optical–SAR sensing.

Abstract

Neural surface reconstruction (NSR) has recently shown strong potential for urban 3D reconstruction from multi-view aerial imagery. However, existing NSR methods often suffer from geometric ambiguity and instability, particularly under sparse-view conditions. This issue is critical in large-scale urban remote sensing, where aerial image acquisition is limited by flight paths, terrain, and cost. To address this challenge, we present the first urban NSR framework that fuses 3D synthetic aperture radar (SAR) point clouds with aerial imagery for high-fidelity reconstruction under constrained, sparse-view settings. 3D SAR can efficiently capture large-scale geometry even from a single side-looking flight path, providing robust priors that complement photometric cues from images. Our framework integrates radar-derived spatial constraints into an SDF-based NSR backbone, guiding structure-aware ray selection and adaptive sampling for stable and efficient optimization. We also construct the first benchmark dataset with co-registered 3D SAR point clouds and aerial imagery, facilitating systematic evaluation of cross-modal 3D reconstruction. Extensive experiments show that incorporating 3D SAR markedly enhances reconstruction accuracy, completeness, and robustness compared with single-modality baselines under highly sparse and oblique-view conditions, highlighting a viable route toward scalable high-fidelity urban reconstruction with advanced airborne and spaceborne optical-SAR sensing.

Figures (10)

• Figure 1: Overview of proposed method. Fusing constrained sparse-view aerial imagery with 3D SAR point clouds provides strong geometric priors for neural surface reconstruction, yielding finer geometry, faster convergence, and higher reconstruction accuracy.
• Figure 2: Proposed method framework. An SDF-based NSR network is optimized under joint supervision from aerial imagery and 3D SAR point clouds, enabling accurate and stable urban surface reconstruction.
• Figure 3: Structure-aware ray selection strategy.
• Figure 4: Geometry-constrained ray bounding strategy.
• Figure 5: Examples from the SARMV3D dataset.