Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance

TL;DR

This work addresses the scarcity of frequent high-quality aerial imagery by proposing GPG2A, a two-stage diffusion-based framework that first predicts a BEV layout from ground views and then synthesizes aerial images conditioned on the BEV map and dynamic text prompts. The BEV prior geometry reduces domain gap and improves geometric fidelity, while text conditioning adds contextual richness to the synthesis. A new multi-modal dataset, VIGORv2, with center-aligned aerial-ground pairs, BEV layouts, and ground-text descriptions, enables robust training and evaluation, including same-area and cross-area protocols. The approach yields superior geometry-preserving aerial synthesis compared to baselines and demonstrates practical utility for cross-view geo-localization augmentation and sketch-based region search, with publicly released code and data.

Abstract

Aerial imagery analysis is critical for many research fields. However, obtaining frequent high-quality aerial images is not always accessible due to its high effort and cost requirements. One solution is to use the Ground-to-Aerial (G2A) technique to synthesize aerial images from easily collectible ground images. However, G2A is rarely studied, because of its challenges, including but not limited to, the drastic view changes, occlusion, and range of visibility. In this paper, we present a novel Geometric Preserving Ground-to-Aerial (G2A) image synthesis (GPG2A) model that can generate realistic aerial images from ground images. GPG2A consists of two stages. The first stage predicts the Bird's Eye View (BEV) segmentation (referred to as the BEV layout map) from the ground image. The second stage synthesizes the aerial image from the predicted BEV layout map and text descriptions of the ground image. To train our model, we present a new multi-modal cross-view dataset, namely VIGORv2 which is built upon VIGOR with newly collected aerial images, maps, and text descriptions. Our extensive experiments illustrate that GPG2A synthesizes better geometry-preserved aerial images than existing models. We also present two applications, data augmentation for cross-view geo-localization and sketch-based region search, to further verify the effectiveness of our GPG2A. The code and data will be publicly available.

Figures (16)

• Figure 1: An example generated aerial image (top right) by our GPG2A from the input text prompt (top left) and the ground image (bottom left). The ground truth aerial image is on the bottom right.
• Figure 2: Left: Aerial image (left), ground image (middle), BEV layout map (right), and text description (bottom) from VIGORv2. Right: The new training (blue lines) and testing (red lines) geographically split of New York City portion of VIGORv2. The non-overlapping training and testing sets prevent data leakage.
• Figure 3: The main architecture of our GPG2A. The first stage is composed of BEV projection and multi-scale layout prediction. Each column in $f_g$ is projected into a polar ray in $f_{BEV}$. The multi-scale network generates the BEV layout map. Then, the second stage synthesizes the aerial image using both $\hat{I}_L$ and the dynamic text prompt. All blocks with a lock symbol indicate a frozen model
• Figure 4: One reference image and three samples for evaluation metrics comparisons.
• Figure 5: Same-area qualitative comparison. From left to right are ground images, target aerial images, ours synthesized BEV layouts and aerial images, ControlNet controlnet, X-seq regmi2018cross, and X-fork regmi2018cross.