Dfilled: Repurposing Edge-Enhancing Diffusion for Guided DSM Void Filling

TL;DR

This work addresses voids in Digital Surface Models (DSMs) derived from stereo imagery, which impede accurate topographic analyses in urban and natural environments. It introduces Dfilled, a guided, diffusion-based inpainting approach that repurposes deep anisotropic diffusion models for DSM restoration, framing the problem with the heat equation $\frac{\partial u}{\partial t} = \nabla \cdot (D(x,y) \nabla u)$ and pursuing a steady-state solution. A two-stage model combines a coarse reconstruction with a local refinement network and a diffusion-based refinement, guided by high-resolution optical imagery, and uses Perlin-noise masks to simulate realistic void patterns for training and evaluation. Across real and synthetic datasets, Dfilled outperforms traditional interpolation and state-of-the-art deep-learning baselines, demonstrating improved edge preservation and visual realism, which is valuable for applications in urban planning, vegetation analysis, and 3D reconstruction. The method's robustness to various mask types and large voids suggests broad practical impact for generating complete, reliable DSMs from incomplete stereo data.

Abstract

Digital Surface Models (DSMs) are essential for accurately representing Earth's topography in geospatial analyses. DSMs capture detailed elevations of natural and manmade features, crucial for applications like urban planning, vegetation studies, and 3D reconstruction. However, DSMs derived from stereo satellite imagery often contain voids or missing data due to occlusions, shadows, and lowsignal areas. Previous studies have primarily focused on void filling for digital elevation models (DEMs) and Digital Terrain Models (DTMs), employing methods such as inverse distance weighting (IDW), kriging, and spline interpolation. While effective for simpler terrains, these approaches often fail to handle the intricate structures present in DSMs. To overcome these limitations, we introduce Dfilled, a guided DSM void filling method that leverages optical remote sensing images through edge-enhancing diffusion. Dfilled repurposes deep anisotropic diffusion models, which originally designed for super-resolution tasks, to inpaint DSMs. Additionally, we utilize Perlin noise to create inpainting masks that mimic natural void patterns in DSMs. Experimental evaluations demonstrate that Dfilled surpasses traditional interpolation methods and deep learning approaches in DSM void filling tasks. Both quantitative and qualitative assessments highlight the method's ability to manage complex features and deliver accurate, visually coherent results.

Figures (6)

• Figure 1: Overview of the proposed guided DSM void filling approach. The method utilizes a image guidance to fill voids in the DSM, resulting in a complete and accurate DSM reconstruction.
• Figure 2: Summary of the proposed architecture for DSM void filling. The method comprises a two-step process: First, high-dimensional features are extracted from both the void-filled DSM and high-resolution optical imagery using a pre-trained feature extractor. Next, a refinement network integrates residual blocks and upsampling operations to reconstruct missing elevation values. This is followed by an edge-enhancing diffusion network that iteratively refines the DSM, leveraging edge features from the optical imagery to ensure accurate and realistic reconstruction of terrain and structural details.
• Figure 3: Real and synthetic void masks for DSM void filling. (a) Real Voids illustrate naturally occurring, complex patterns in DSMs. (b) LaMa masks are structured synthetic voids often used in model training but may not accurately reflect real void distributions. (c) Perlin masks, generated with procedural noise, better mimic the irregularity and complexity of real voids.
• Figure 4: Visual comparison of void filling results for small masks, where voids cover a relatively small portion of the DSM. The proposed Dfilled method produces more regularized and smoother results compared to RSAGAN, aligning closely with the ground truth while preserving fine-scale structural details. Green boxes indicate regions of interest.
• Figure 5: Visual comparison of void filling results for large masks, where 60–80% of the DSM area is void-filled. The Dfilled method effectively utilizes high-resolution guide images to reconstruct missing elevation data, outperforming RSAGAN in recovering complex terrain features and structural details.