Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task

Hannuo Zhang, Huihui Li, Jiarui Lin, Yujie Zhang, Jianghua Fan, Hang Liu

TL;DR

This work tackles SAR-to-optical image translation for ship targets under conditions where optical imagery is challenging. It introduces Seg-CycleGAN, a downstream-task-guided framework that leverages a pre-trained ship segmentation module (trained with SAM-derived labels) to steer the GAN-based translation, preserving ship semantics in the optical domain. Experimental results on HRSID-DIOR and WHU-OPT-SAR show improved segmentation-oriented metrics and competitive image quality, highlighting the value of foundation-model-annotated data for SAR-to-optical translation. The approach suggests a broader paradigm where downstream tasks guide image translation, with practical implications for robust ship monitoring and related earth-observation applications.

Abstract

Optical remote sensing and Synthetic Aperture Radar(SAR) remote sensing are crucial for earth observation, offering complementary capabilities. While optical sensors provide high-quality images, they are limited by weather and lighting conditions. In contrast, SAR sensors can operate effectively under adverse conditions. This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN, designed to enhance the accuracy of ship target translation by leveraging semantic information from a pre-trained semantic segmentation model. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network, improving the quality of output Optical-styled images. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed. This work suggests broader research and applications for downstream-task-guided frameworks. The code will be available at https://github.com/NPULHH/

Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task

TL;DR

This work tackles SAR-to-optical image translation for ship targets under conditions where optical imagery is challenging. It introduces Seg-CycleGAN, a downstream-task-guided framework that leverages a pre-trained ship segmentation module (trained with SAM-derived labels) to steer the GAN-based translation, preserving ship semantics in the optical domain. Experimental results on HRSID-DIOR and WHU-OPT-SAR show improved segmentation-oriented metrics and competitive image quality, highlighting the value of foundation-model-annotated data for SAR-to-optical translation. The approach suggests a broader paradigm where downstream tasks guide image translation, with practical implications for robust ship monitoring and related earth-observation applications.

Abstract

Optical remote sensing and Synthetic Aperture Radar(SAR) remote sensing are crucial for earth observation, offering complementary capabilities. While optical sensors provide high-quality images, they are limited by weather and lighting conditions. In contrast, SAR sensors can operate effectively under adverse conditions. This letter proposes a GAN-based SAR-to-optical image translation method named Seg-CycleGAN, designed to enhance the accuracy of ship target translation by leveraging semantic information from a pre-trained semantic segmentation model. Our method utilizes the downstream task of ship target semantic segmentation to guide the training of image translation network, improving the quality of output Optical-styled images. The potential of foundation-model-annotated datasets in SAR-to-optical translation tasks is revealed. This work suggests broader research and applications for downstream-task-guided frameworks. The code will be available at https://github.com/NPULHH/
Paper Structure (14 sections, 5 equations, 5 figures, 3 tables)

This paper contains 14 sections, 5 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Labeling Datasets with SAM
  4. Network Architectures
  5. Loss Functions
  6. Training Details
  7. Experiment Results
  8. Datasets
  9. Qualitative Evaluation
  10. Quantitative Evaluation
  11. Conclusion
  12. Appendix
  13. Visualization of experiments on WHU-OPT-SAR
  14. Visualization of SAM-annotated datasets

Figures (5)

  • Figure 1: Structure of Seg-CycleGAN for SAR-to-optical image translation.
  • Figure 2: Examples of ship target translation for samples in HRSID
  • Figure 3: Examples of image translation for samples in WHU-OPT-SAR
  • Figure 4: Visualization of SAM-annotated samples in HRSC2016-MS
  • Figure 5: Visualization of SAM-annotated samples in DIOR