Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Remote Sensing Foundation Models: From Vision to Multimodality

Ziyue Huang, Hongxi Yan, Qiqi Zhan, Shuai Yang, Mingming Zhang, Chenkai Zhang, YiMing Lei, Zeming Liu, Qingjie Liu, Yunhong Wang

TL;DR

The paper surveys remote sensing foundation models, detailing architecture (vision-only and multimodal), training paradigms (contrastive, generative, and hybrid), data sources (custom and public RS datasets), and evaluation across a broad task spectrum. It introduces a two-dimensional taxonomy (model architecture and primary functionality) to organize progress and highlight how RS data characteristics shape model design, training, and deployment. Key contributions include a comprehensive taxonomy, a resource repository, and synthesis of benchmarks and results that expose strengths and limitations of current RS foundation models. The work underscores practical implications for scalable, cross-modal geospatial understanding and points to directions such as multimodal fusion, geographic knowledge integration, and efficient architectures to push real-world impact. Overall, the survey clarifies the state of the field and provides a roadmap for advancing robust, generalizable RS foundation models in diverse environments.

Abstract

The rapid advancement of remote sensing foundation models, particularly vision and multimodal models, has significantly enhanced the capabilities of intelligent geospatial data interpretation. These models combine various data modalities, such as optical, radar, and LiDAR imagery, with textual and geographic information, enabling more comprehensive analysis and understanding of remote sensing data. The integration of multiple modalities allows for improved performance in tasks like object detection, land cover classification, and change detection, which are often challenged by the complex and heterogeneous nature of remote sensing data. However, despite these advancements, several challenges remain. The diversity in data types, the need for large-scale annotated datasets, and the complexity of multimodal fusion techniques pose significant obstacles to the effective deployment of these models. Moreover, the computational demands of training and fine-tuning multimodal models require significant resources, further complicating their practical application in remote sensing image interpretation tasks. This paper provides a comprehensive review of the state-of-the-art in vision and multimodal foundation models for remote sensing, focusing on their architecture, training methods, datasets and application scenarios. We discuss the key challenges these models face, such as data alignment, cross-modal transfer learning, and scalability, while also identifying emerging research directions aimed at overcoming these limitations. Our goal is to provide a clear understanding of the current landscape of remote sensing foundation models and inspire future research that can push the boundaries of what these models can achieve in real-world applications. The list of resources collected by the paper can be found in the https://github.com/IRIP-BUAA/A-Review-for-remote-sensing-vision-language-models.

A Survey on Remote Sensing Foundation Models: From Vision to Multimodality

TL;DR

The paper surveys remote sensing foundation models, detailing architecture (vision-only and multimodal), training paradigms (contrastive, generative, and hybrid), data sources (custom and public RS datasets), and evaluation across a broad task spectrum. It introduces a two-dimensional taxonomy (model architecture and primary functionality) to organize progress and highlight how RS data characteristics shape model design, training, and deployment. Key contributions include a comprehensive taxonomy, a resource repository, and synthesis of benchmarks and results that expose strengths and limitations of current RS foundation models. The work underscores practical implications for scalable, cross-modal geospatial understanding and points to directions such as multimodal fusion, geographic knowledge integration, and efficient architectures to push real-world impact. Overall, the survey clarifies the state of the field and provides a roadmap for advancing robust, generalizable RS foundation models in diverse environments.

Abstract

The rapid advancement of remote sensing foundation models, particularly vision and multimodal models, has significantly enhanced the capabilities of intelligent geospatial data interpretation. These models combine various data modalities, such as optical, radar, and LiDAR imagery, with textual and geographic information, enabling more comprehensive analysis and understanding of remote sensing data. The integration of multiple modalities allows for improved performance in tasks like object detection, land cover classification, and change detection, which are often challenged by the complex and heterogeneous nature of remote sensing data. However, despite these advancements, several challenges remain. The diversity in data types, the need for large-scale annotated datasets, and the complexity of multimodal fusion techniques pose significant obstacles to the effective deployment of these models. Moreover, the computational demands of training and fine-tuning multimodal models require significant resources, further complicating their practical application in remote sensing image interpretation tasks. This paper provides a comprehensive review of the state-of-the-art in vision and multimodal foundation models for remote sensing, focusing on their architecture, training methods, datasets and application scenarios. We discuss the key challenges these models face, such as data alignment, cross-modal transfer learning, and scalability, while also identifying emerging research directions aimed at overcoming these limitations. Our goal is to provide a clear understanding of the current landscape of remote sensing foundation models and inspire future research that can push the boundaries of what these models can achieve in real-world applications. The list of resources collected by the paper can be found in the https://github.com/IRIP-BUAA/A-Review-for-remote-sensing-vision-language-models.

Paper Structure

This paper contains 45 sections, 4 equations, 4 figures, 22 tables.

Table of Contents

  1. Introduction
  2. Model Architecture
  3. Vision Foundation Model
  4. CNN-based Model
  5. Transformer-based Model
  6. CNN-Transfromer Hybrid Model
  7. Multimodal Foundation Models
  8. CLIP-based Model
  9. Diffusion-based Model
  10. Multimodal Large Language Model
  11. LLM-based Agent
  12. Training Methods
  13. Vision Foundation Model
  14. Contrastive Method
  15. Generative Method
  16. ...and 30 more sections

Figures (4)

  • Figure 1: Organization and Taxonomy of this article.
  • Figure 2: The architectures of remote sensing multimodal foundation models, primarily includes CLIP-based model, MLLM, and diffusion-based model. Agent models are built upon the LLM architecture.
  • Figure 3: Training methods for Remote Sensing Vision Foundation Models.
  • Figure 4: Contrastive and generative training methods for vision foundation model. Existing studies refine these methods by incorporating adaptations specific to remote sensing scenarios.