EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation

TL;DR

EarthMapper introduces a geo-conditioned autoregressive framework that unifies bidirectional satellite-map translation via geo-aware joint-scale autoregression and hierarchical residual quantization. The model is enhanced by semantic infusion and a dynamic complexity-guided inference scheme, plus a key-point force mechanism to balance fidelity and diversity. The CNSatMap dataset (302,132 pairs across 38 Chinese cities) provides a robust benchmark for cross-modal geospatial translation, with strong results on New York as well. Zero-shot capabilities in in-painting, out-painting, and coordinate-conditional generation demonstrate strong generalization and practical applicability for urban planning and disaster response.

Abstract

Satellite imagery and maps, as two fundamental data modalities in remote sensing, offer direct observations of the Earth's surface and human-interpretable geographic abstractions, respectively. The task of bidirectional translation between satellite images and maps (BSMT) holds significant potential for applications in urban planning and disaster response. However, this task presents two major challenges: first, the absence of precise pixel-wise alignment between the two modalities substantially complicates the translation process; second, it requires achieving both high-level abstraction of geographic features and high-quality visual synthesis, which further elevates the technical complexity. To address these limitations, we introduce EarthMapper, a novel autoregressive framework for controllable bidirectional satellite-map translation. EarthMapper employs geographic coordinate embeddings to anchor generation, ensuring region-specific adaptability, and leverages multi-scale feature alignment within a geo-conditioned joint scale autoregression (GJSA) process to unify bidirectional translation in a single training cycle. A semantic infusion (SI) mechanism is introduced to enhance feature-level consistency, while a key point adaptive guidance (KPAG) mechanism is proposed to dynamically balance diversity and precision during inference. We further contribute CNSatMap, a large-scale dataset comprising 302,132 precisely aligned satellite-map pairs across 38 Chinese cities, enabling robust benchmarking. Extensive experiments on CNSatMap and the New York dataset demonstrate EarthMapper's superior performance, achieving significant improvements in visual realism, semantic consistency, and structural fidelity over state-of-the-art methods. Additionally, EarthMapper excels in zero-shot tasks like in-painting, out-painting and coordinate-conditional generation, underscoring its versatility.

Figures (10)

• Figure 1: Conceptual illustration of bidirectional satellite-map translation and their respective applications.
• Figure 2: Illustration of the geographical distribution of satellite-map pairs sampled from the proposed CNSatMap dataset.
• Figure 3: Overview of our proposed EarthMapper framework, with the upper section dedicated to training and the lower section to inference. During training, paired satellite and map data are initially processed by a frozen image encoder and quantized into vector representations using the hierarchical residual quantization (HRQ) module. Concurrently, geographic coordinates are encoded as initial vectors and fed into the geo-conditioned joint scale autoregression (GJSA) model, which generates image content by progressively increasing the resolution. The resulting output is then passed to the semantic infusion (SI) module, where it is aligned with ground truth features to improve the realism of the generated image, before being decoded into an image using the VAE decoder. In the inference phase, for instance, when generating satellite images from maps, the map is quantized into vectors, combined with geographic coordinate vectors, and input into the model. Key points are computed using the key point force (KPF) method and incorporated into the generation process, while complexity guidance (CG) dynamically modulates the control intensity. Finally, the output is decoded to produce the satellite image.
• Figure 4: Schematic diagram of the inference section, with keypoint adaptive forcing on the left and complexity bootstrapping on the right, both of which are connected by the computation of complexity and together comprise the inference.
• Figure 5: Qualitative comparison of bidirectional satellite-map translation results on the New York test set. The top five rows illustrate map-to-satellite translation, and the bottom five rows depict satellite-to-map translation.