SARVLM: A Vision Language Foundation Model for Semantic Understanding and Target Recognition in SAR Imagery

TL;DR

SARVLM presents a dedicated vision-language foundation for SAR imagery by combining SARCLIP and SARCap and by constructing SARVLM-1M, a large-scale image-text dataset with 1.7M pairs. A two-stage domain transfer from optical to SAR enables cross-modal semantic alignment, yielding strong retrieval and target recognition performance, plus capable zero-shot and captioning capabilities. The work demonstrates significant gains over optical baselines on SAR-centric tasks and provides a robust foundation for SAR semantic understanding, with potential for further integration with multi-modal LLMs in the future.

Abstract

Synthetic Aperture Radar (SAR) is a crucial imaging modality thanks to its all-weather capability. Although recent advances in self-supervised learning and masked image modeling (MIM) have enabled SAR foundation models, these methods largely emphasize low-level visual features and often overlook multimodal alignment and zero-shot target recognition in SAR imagery. To address this, we construct SARVLM-1M, a large-scale vision-language dataset with over one million image-text pairs aggregated from existing datasets. We further propose a domain transfer training strategy to mitigate the large gap between natural and SAR imagery. Building on this, we develop SARVLM, the first vision language foundation model (VLM) tailored to SAR, comprising SARCLIP and SARCap. SARVLM is trained with a vision-language contrastive objective under the proposed domain transfer strategy, bridging SAR imagery and textual descriptions. Extensive experiments on image text retrieval, zero-shot classification, semantic localization, and imagery captioning demonstrate that SARVLM delivers superior feature extraction and interpretation, outperforming state-of-the-art VLMs and advancing SAR semantic understanding. Code and datasets will be released soon.

Figures (9)

• Figure 1: Comparison of image-text datasets in remote sensing. Gray circles represent datasets in the optical image domain, blue circles denote those in the SAR domain, and the red circle indicates our proposed SARVLM-1M. In the SAR domain, SARVLM-1M significantly surpasses existing datasets in both image and text volume.
• Figure 2: The paradigm for foundation model: (a) CL-based methods, (b) MIM-based methods, (c) CLIP-based methods.
• Figure 3: (a) Examples from the SARVLM-1M dataset; (b) Two-stage domain transfer training strategy for SARCLIP.
• Figure 4: The framework of SARCap method.
• Figure 5: The word cloud of SARVLM-1M dataset.