TerraGen: A Unified Multi-Task Layout Generation Framework for Remote Sensing Data Augmentation

TL;DR

TerraGen tackles the data bottleneck and task-isolation in remote sensing by introducing a diffusion-based, multi-conditional framework that unifies layout and text conditioning for several RS tasks. It combines a geographic-spatial layout encoder, adaptive task conditioning, and a multi-scale injection strategy to achieve pixel-accurate, geographically coherent synthesis across detection, segmentation, building/road extraction, and flood mapping. The authors provide the first large-scale multi-task RS layout dataset (≈45k samples) and a standardized evaluation benchmark, showing state-of-the-art image quality and meaningful improvements in downstream task performance, including in few-shot regimes. This work demonstrates that spatial layouts can serve as a universal representation bridging diverse RS tasks, enabling efficient cross-task data augmentation and knowledge transfer with practical impact for remote sensing pipelines.

Abstract

Remote sensing vision tasks require extensive labeled data across multiple, interconnected domains. However, current generative data augmentation frameworks are task-isolated, i.e., each vision task requires training an independent generative model, and ignores the modeling of geographical information and spatial constraints. To address these issues, we propose \textbf{TerraGen}, a unified layout-to-image generation framework that enables flexible, spatially controllable synthesis of remote sensing imagery for various high-level vision tasks, e.g., detection, segmentation, and extraction. Specifically, TerraGen introduces a geographic-spatial layout encoder that unifies bounding box and segmentation mask inputs, combined with a multi-scale injection scheme and mask-weighted loss to explicitly encode spatial constraints, from global structures to fine details. Also, we construct the first large-scale multi-task remote sensing layout generation dataset containing 45k images and establish a standardized evaluation protocol for this task. Experimental results show that our TerraGen can achieve the best generation image quality across diverse tasks. Additionally, TerraGen can be used as a universal data-augmentation generator, enhancing downstream task performance significantly and demonstrating robust cross-task generalisation in both full-data and few-shot scenarios.

Figures (6)

• Figure 1: Generation results of the TerraGen model and composition of the TerraGen dataset. The main visuals display high-fidelity remote sensing images synthesized by our TerraGen model from given layout conditions. The pie chart in the top-right corner illustrates the five-task distribution of our purpose-built TerraGen dataset.
• Figure 2: An overview of the TerraGen architecture. TerraGen utilizes a multi-conditional generation framework that integrates geographic-spatial layout encoding with multi-scale injection mechanisms. The system processes task specifications, textual descriptions, and spatial layouts through mask-cross-attention blocks and injects layout guidance at multiple U-Net resolution levels to ensure both global layout consistency and fine-grained spatial accuracy in RS image generation.
• Figure 3: Word cloud of semantic categories covered in our dataset.
• Figure 4: Qualitative comparison for mask-to-image (top four rows) and bbox-to-image (bottom row) generation by various methods.
• Figure 5: TerraGen Generation Showcase: Images synthesized from conditional layouts.