Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SAT-NGP : Unleashing Neural Graphics Primitives for Fast Relightable Transient-Free 3D reconstruction from Satellite Imagery

Camille Billouard, Dawa Derksen, Emmanuelle Sarrazin, Bruno Vallet

TL;DR

SAT-NGP tackles the challenge of building accurate 3D reconstructions from multi-date satellite imagery where illumination, shadows, and transient objects cause inconsistencies. It combines a multi-resolution hash encoding inspired by Instant Neural Graphics Primitives with a lightweight NeRF architecture, solar-angle conditioning via SH, a robust loss for transient suppression, and UTMs-based scene representation to achieve relightable, transient-free DSMs. The approach delivers dramatic speedups (training in about $15$ minutes) while maintaining DSM quality comparable to slower NeRF variants, and it removes transient objects from the novel-view synthesis. The work enables scalable, high-quality 3D reconstruction over large satellite datasets on modest GPU resources, with code to be released.

Abstract

Current stereo-vision pipelines produce high accuracy 3D reconstruction when using multiple pairs or triplets of satellite images. However, these pipelines are sensitive to the changes between images that can occur as a result of multi-date acquisitions. Such variations are mainly due to variable shadows, reflexions and transient objects (cars, vegetation). To take such changes into account, Neural Radiance Fields (NeRF) have recently been applied to multi-date satellite imagery. However, Neural methods are very compute-intensive, taking dozens of hours to learn, compared with minutes for standard stereo-vision pipelines. Following the ideas of Instant Neural Graphics Primitives we propose to use an efficient sampling strategy and multi-resolution hash encoding to accelerate the learning. Our model, Satellite Neural Graphics Primitives (SAT-NGP) decreases the learning time to 15 minutes while maintaining the quality of the 3D reconstruction.

SAT-NGP : Unleashing Neural Graphics Primitives for Fast Relightable Transient-Free 3D reconstruction from Satellite Imagery

TL;DR

SAT-NGP tackles the challenge of building accurate 3D reconstructions from multi-date satellite imagery where illumination, shadows, and transient objects cause inconsistencies. It combines a multi-resolution hash encoding inspired by Instant Neural Graphics Primitives with a lightweight NeRF architecture, solar-angle conditioning via SH, a robust loss for transient suppression, and UTMs-based scene representation to achieve relightable, transient-free DSMs. The approach delivers dramatic speedups (training in about minutes) while maintaining DSM quality comparable to slower NeRF variants, and it removes transient objects from the novel-view synthesis. The work enables scalable, high-quality 3D reconstruction over large satellite datasets on modest GPU resources, with code to be released.

Abstract

Current stereo-vision pipelines produce high accuracy 3D reconstruction when using multiple pairs or triplets of satellite images. However, these pipelines are sensitive to the changes between images that can occur as a result of multi-date acquisitions. Such variations are mainly due to variable shadows, reflexions and transient objects (cars, vegetation). To take such changes into account, Neural Radiance Fields (NeRF) have recently been applied to multi-date satellite imagery. However, Neural methods are very compute-intensive, taking dozens of hours to learn, compared with minutes for standard stereo-vision pipelines. Following the ideas of Instant Neural Graphics Primitives we propose to use an efficient sampling strategy and multi-resolution hash encoding to accelerate the learning. Our model, Satellite Neural Graphics Primitives (SAT-NGP) decreases the learning time to 15 minutes while maintaining the quality of the 3D reconstruction.
Paper Structure (15 sections, 4 equations, 3 figures, 1 table)

This paper contains 15 sections, 4 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related works
  3. NeRF for Satellite imagery
  4. NeRF Training and Inference acceleration
  5. METHODOLOGY
  6. Encoding
  7. Architecture
  8. Loss
  9. Implementation details
  10. EXPERIMENTS AND ANALYSIS
  11. Datasets and ground truth details
  12. Evaluation Metrics
  13. Results
  14. CONCLUSION
  15. ACKNOWLEDGEMENTS

Figures (3)

  • Figure 1: Compared to the ground truth LiDAR and unseen WorldView-3 image (top), SAT-NGP (middle) yields more accurate surface extraction and similar novel views in 14 minutes, as SAT-NeRF (bottom) in 12 hours.
  • Figure 2: Comparison between ground truth unseen view (left column) and NVS (right column). We show transient-free (top two row) and relightable capabilities on NVS.
  • Figure 3: MAE evolution during the first 5 epochs with the training time of each one of SAT-NeRF compared to our method.