Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery

TL;DR

Skyfall-GS tackles the challenge of generating large-scale, navigable 3D urban environments without ground-truth 3D scans by fusing satellite-derived coarse geometry with diffusion-based texture enhancement. It introduces a two-stage pipeline: (i) Reconstruction using 3D Gaussian Splatting with appearance modeling and pseudo-depth supervision to produce a geometrically faithful base, and (ii) Synthesis via curriculum-guided Iterative Dataset Update (IDU) that leverages pre-trained diffusion priors (FlowEdit) to hallucinate occluded facades and refine textures while preserving satellite geometry. The approach demonstrates superior cross-view consistency and perceptual realism on diverse datasets (DFC2019 and GoogleEarth) and runs in real time, enabling immersive exploration without ground-level data. Key contributions include the first city-block scale framework that avoids fixed-domain 3D annotations, diffusion-prior-based appearance refinement, and curriculum-based view progression to robustly recover occluded details across multi-date satellite imagery.

Abstract

Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose \textbf{Skyfall-GS}, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/

Figures (15)

• Figure 1: Our method synthesizes high-quality, immersive 3D urban scenes solely from multi-view satellite imagery, enabling realistic drone-view navigation without relying on additional 3D or street-level training data. Given multiple satellite images from diverse viewpoints and dates (left), our method leverages 3D Gaussian Splatting combined with pre-trained text-to-image diffusion models in an iterative refinement framework to generate realistic 3D block-scale city from limited satellite-view input (right). Our method significantly enhances visual fidelity, geometric sharpness, and semantic consistency, enabling real-time immersive exploration.
• Figure 2: Limitations of existing novel-view synthesis methods from satellite imagery. (a) Sat-NeRF mari2022sat and naive 3DGS kerbl20233d yield blurred or distorted building facades due to insufficient geometric detail and limited parallax from satellite viewpoints. (b) City generation methods xie2024citydreamerxie2025gaussiancity produce oversimplified building geometries and unrealistic appearances, primarily due to strong assumptions about the input data, and overfitting to small-scale, domain-specific datasets. In comparison, our method synthesizes more realistic appearances and geometries from aerial views.
• Figure 3: Overview of the proposed Skyfall-GS pipeline. Our method synthesizes immersive and free-flight navigable city-block scale 3D scenes solely from multi-view satellite imagery in two stages. (a) In the Reconstruction Stage, we first reconstruct the initial 3D scene using 3DGS, enhanced by pseudo-camera depth supervision to address limited parallax in satellite images. We use an appearance modeling component to handle varying illumination conditions across multi-date satellite images. (b) In the Synthesis Stage, we introduce a curriculum-based Iterative Dataset Update (IDU) refinement technique leveraging (c) a pre-trained T2I diffusion model flux2024 with prompt-to-prompt editing kulikov2024flowedit. By iteratively updating training datasets with progressively refined renders, our approach significantly reduces visual artifacts, improving geometric accuracy and texture realism, particularly in previously occluded areas such as building facades.
• Figure 4: The motivation of curriculum strategy. Renderings of the initial 3D reconstruction from varied elevation angles reveal progressive degradation as the viewing angle decreases.
• Figure 5: Render refinement. (a) Original 3DGS render with artifacts and blurry textures; (b) Refined result showing enhanced geometry and texture quality.