Multi-view 3D surface reconstruction from SAR images by inverse rendering

TL;DR

This work addresses 3D surface reconstruction from unconstrained SAR images by introducing an inverse rendering framework that uses a differentiable SAR image formation model operating on surface-based geometry. By rasterizing on the DSM and a backscatter map, the method adopts a NeRF-inspired optimization but avoids full volume rendering, enabling efficient, gradient-based recovery of elevation and appearance from a handful of SAR views. A coarse-to-fine training regime with an MLP representation of elevation and backscatter demonstrates the capability to recover accurate surface models and to distinguish geometry from texture on synthetic data generated by EMPRISE. The approach paves the way for multi-sensor data fusion and more flexible SAR 3D reconstruction beyond traditional interferometric constraints.

Abstract

3D reconstruction of a scene from Synthetic Aperture Radar (SAR) images mainly relies on interferometric measurements, which involve strict constraints on the acquisition process. These last years, progress in deep learning has significantly advanced 3D reconstruction from multiple views in optical imaging, mainly through reconstruction-by-synthesis approaches pioneered by Neural Radiance Fields. In this paper, we propose a new inverse rendering method for 3D reconstruction from unconstrained SAR images, drawing inspiration from optical approaches. First, we introduce a new simplified differentiable SAR rendering model, able to synthesize images from a digital elevation model and a radar backscattering coefficients map. Then, we introduce a coarse-to-fine strategy to train a Multi-Layer Perceptron (MLP) to fit the height and appearance of a given radar scene from a few SAR views. Finally, we demonstrate the surface reconstruction capabilities of our method on synthetic SAR images produced by ONERA's physically-based EMPRISE simulator. Our method showcases the potential of exploiting geometric disparities in SAR images and paves the way for multi-sensor data fusion.

Figures (4)

• Figure 1: Iterative determination of shadow areas: (a) Estimation of $v_3$, indicatrix of the shadow of sample $P_3$ knowing $h_2=\eta_2$, the negative slope between the antenna and $P_2$, the latest illuminated sample. (b) Estimation of $v_4$, indicatrix of the shadow of sample $P_4$ knowing $h_3=\eta_2$ from the previous iteration.
• Figure 2: Architecture of the fully-connected network. Input: $(x,y)$, a position on the ground, encoded with is Instant-NGP's positional encoding $\gamma_P$mueller_instant_2022. Outputs: the elevation $\mathcal{Z}(x,y)$ and backscattering coefficient $\mathcal{B}(x,y)$.
• Figure 3: Results of the DSM reconstruction on the low resolution Forez scene. Areas not visible in both images are masked out on the DSM reconstructed from ascending/descending passes (third column).
• Figure 4: Results of the surface learning on the high resolution island scene with multiple materials. Reference scattering coefficients are estimated from the mangia model by an evaluation with 45° incidence.