VectorSynth: Fine-Grained Satellite Image Synthesis with Structured Semantics

TL;DR

VectorSynth tackles the limitation of coarse-region conditioning in satellite image synthesis by learning dense polygon-level alignment between OpenStreetMap tags and imagery through the COSA vision-language model. It then couples this grounding with a ControlNet-based diffusion generator to produce region-specific imagery conditioned on both global text prompts and per-pixel semantic maps, enabling fine-grained edits and interactive what-if scenarios. The approach is validated on a new OSM-Satellite dataset across multiple cities, showing improved semantic fidelity, spatial realism, and open-vocabulary grounding compared with baselines. The work offers practical gains for urban planning, data generation for remote sensing tasks, and semantically-aware map-informed content creation, with a dataset and code released for reproducibility.

Abstract

We introduce VectorSynth, a diffusion-based framework for pixel-accurate satellite image synthesis conditioned on polygonal geographic annotations with semantic attributes. Unlike prior text- or layout-conditioned models, VectorSynth learns dense cross-modal correspondences that align imagery and semantic vector geometry, enabling fine-grained, spatially grounded edits. A vision language alignment module produces pixel-level embeddings from polygon semantics; these embeddings guide a conditional image generation framework to respect both spatial extents and semantic cues. VectorSynth supports interactive workflows that mix language prompts with geometry-aware conditioning, allowing rapid what-if simulations, spatial edits, and map-informed content generation. For training and evaluation, we assemble a collection of satellite scenes paired with pixel-registered polygon annotations spanning diverse urban scenes with both built and natural features. We observe strong improvements over prior methods in semantic fidelity and structural realism, and show that our trained vision language model demonstrates fine-grained spatial grounding. The code and data are available at https://github.com/mvrl/VectorSynth.

Figures (16)

• Figure 1: VectorSynth logo synthesized using learned OSM‑based pixel embeddings. Each letter is generated with distinct OSM tag combinations (e.g. industrial, farmland, geological features), demonstrating VectorSynth’s fine‑grained semantic control over satellite‑image synthesis.
• Figure 2: VectorSynth enables precise, fine-grained control over both spatial location and semantic content during satellite image synthesis. top: Image-level conditioning lacks the ability to target specific regions or object types. bottom: VectorSynth allows conditioning on individual polygons with semantic labels (e.g., $c$ = “school”), producing coherent imagery that respects both spatial extent and semantics.
• Figure 3: Illustration of pixel-level tag assignments. Each pixel inherits tags from overlapping polygons, resulting in compositional tag lists used for downstream learning tasks.
• Figure 4: Architecture overview showing dual encoders for satellite imagery and OSM tag descriptions, with polygon-guided average pooling to extract region-specific embeddings. We align polygon embeddings with grounded OSM tags, enabling fine-grained spatial conditioning for satellite image synthesis.
• Figure 5: This figure presents our semantic-guided image synthesis pipeline which employs a pretrained text encoder to generate dense pixel-level control from input vector geometry.