Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model

Chenyang Liu, Keyan Chen, Rui Zhao, Zhengxia Zou, Zhenwei Shi

TL;DR

This work tackles the gap in global-scale, text-driven remote sensing image generation by introducing Git-10M, a 10.5M image-text dataset with geolocation and resolution metadata, and Text2Earth, a 1.3B parameter diffusion foundation model. Text2Earth uses VAE-based image compression, OpenCLIP ViT-H text embeddings, and a resolution-guided diffusion process with cross-attention and a dynamic conditioning strategy to support zero-shot text-to-image generation, unbounded scene construction, and cross-modal synthesis. The model achieves notable improvements on RSICD benchmarks (e.g., +26.23 in FID and +20.95% in Zero-shot Cls-OA) and demonstrates robust capabilities in image editing, multi-modal generation, and data augmentation, with a scalable training paradigm on a global RS dataset. These contributions enable broader, semantically consistent remote sensing image generation and processing, with potential impacts on simulation, planning, and multimodal RS learning.

Abstract

Generative foundation models have advanced large-scale text-driven natural image generation, becoming a prominent research trend across various vertical domains. However, in the remote sensing field, there is still a lack of research on large-scale text-to-image (text2image) generation technology. Existing remote sensing image-text datasets are small in scale and confined to specific geographic areas and scene types. Besides, existing text2image methods have struggled to achieve global-scale, multi-resolution controllable, and unbounded image generation. To address these challenges, this paper presents two key contributions: the Git-10M dataset and the Text2Earth foundation model. Git-10M is a global-scale image-text dataset comprising 10.5 million image-text pairs, 5 times larger than the previous largest one. The dataset covers a wide range of geographic scenes and contains resolution information, significantly surpassing existing datasets in both size and diversity. Building on Git-10M, we propose Text2Earth, a 1.3 billion parameter generative foundation model based on the diffusion framework to model global-scale remote sensing scenes. Text2Earth integrates a resolution guidance mechanism, enabling users to specify image resolutions. A dynamic condition adaptation strategy is proposed for training and inference to improve image quality. Text2Earth excels in zero-shot text2image generation and demonstrates robust generalization and flexibility across multiple tasks, including unbounded scene construction, image editing, and cross-modal image generation. This robust capability surpasses previous models restricted to the basic fixed size and limited scene types. On the previous benchmark dataset, Text2Earth outperforms previous models with an improvement of +26.23 FID and +20.95% Zero-shot Cls-OA metric.Our project page is https://chen-yang-liu.github.io/Text2Earth

Text2Earth: Unlocking Text-driven Remote Sensing Image Generation with a Global-Scale Dataset and a Foundation Model

TL;DR

This work tackles the gap in global-scale, text-driven remote sensing image generation by introducing Git-10M, a 10.5M image-text dataset with geolocation and resolution metadata, and Text2Earth, a 1.3B parameter diffusion foundation model. Text2Earth uses VAE-based image compression, OpenCLIP ViT-H text embeddings, and a resolution-guided diffusion process with cross-attention and a dynamic conditioning strategy to support zero-shot text-to-image generation, unbounded scene construction, and cross-modal synthesis. The model achieves notable improvements on RSICD benchmarks (e.g., +26.23 in FID and +20.95% in Zero-shot Cls-OA) and demonstrates robust capabilities in image editing, multi-modal generation, and data augmentation, with a scalable training paradigm on a global RS dataset. These contributions enable broader, semantically consistent remote sensing image generation and processing, with potential impacts on simulation, planning, and multimodal RS learning.

Abstract

Generative foundation models have advanced large-scale text-driven natural image generation, becoming a prominent research trend across various vertical domains. However, in the remote sensing field, there is still a lack of research on large-scale text-to-image (text2image) generation technology. Existing remote sensing image-text datasets are small in scale and confined to specific geographic areas and scene types. Besides, existing text2image methods have struggled to achieve global-scale, multi-resolution controllable, and unbounded image generation. To address these challenges, this paper presents two key contributions: the Git-10M dataset and the Text2Earth foundation model. Git-10M is a global-scale image-text dataset comprising 10.5 million image-text pairs, 5 times larger than the previous largest one. The dataset covers a wide range of geographic scenes and contains resolution information, significantly surpassing existing datasets in both size and diversity. Building on Git-10M, we propose Text2Earth, a 1.3 billion parameter generative foundation model based on the diffusion framework to model global-scale remote sensing scenes. Text2Earth integrates a resolution guidance mechanism, enabling users to specify image resolutions. A dynamic condition adaptation strategy is proposed for training and inference to improve image quality. Text2Earth excels in zero-shot text2image generation and demonstrates robust generalization and flexibility across multiple tasks, including unbounded scene construction, image editing, and cross-modal image generation. This robust capability surpasses previous models restricted to the basic fixed size and limited scene types. On the previous benchmark dataset, Text2Earth outperforms previous models with an improvement of +26.23 FID and +20.95% Zero-shot Cls-OA metric.Our project page is https://chen-yang-liu.github.io/Text2Earth
Paper Structure (37 sections, 11 equations, 15 figures, 6 tables, 2 algorithms)

This paper contains 37 sections, 11 equations, 15 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Foundation Models in the Computer Vision
  4. GANs-Based Models
  5. Autoregressive Models
  6. Diffusion-Based Models
  7. Remote Sensing Text2Image Generation
  8. Global-scale image-text Dataset
  9. Image Collection and Preprocessing
  10. Text Annotation
  11. Dataset Analysis
  12. Text2Earth Foundation Model
  13. Structure of Text2Earth Model
  14. Image Compression Encoding
  15. Diffusion Modeling
  16. ...and 22 more sections

Figures (15)

  • Figure 1: Comparison between previous remote sensing text2image datasets and our Git-10M dataset.
  • Figure 2: The diverse image composition of the Git-10M dataset. Most images were collected from Google Earth, allowing public sharing and redistribution.
  • Figure 3: The diverse geospatial distribution of the Git-10M dataset. The yellow pixels represent the geographic locations where remote sensing images in Git-10M were sampled. The distribution shows that our dataset covers multiple continents and geographical regions, covering various typical scenes such as urban areas, forests, mountains, and deserts.
  • Figure 4: The distribution of images with varying resolutions in the Git-10M dataset. The dataset encompasses images ranging from high resolution (e.g., 0.5m/pixel) to low resolution (e.g., 128m/pixel).
  • Figure 6: Text Analysis. Top: the word cloud of the texts in the Git-10M dataset. Bottom: the distribution of text lengths in the Git-10M dataset shows that each textual description averages approximately 52 words, with the entire dataset comprising over 10.5 million text samples and more than 5.5 billion words.
  • ...and 10 more figures