Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

M$^2$CD: A Unified MultiModal Framework for Optical-SAR Change Detection with Mixture of Experts and Self-Distillation

Ziyuan Liu, Jiawei Zhang, Wenyu Wang, Yuantao Gu

TL;DR

This paper addresses optical-SAR change detection by bridging the modality gap that hampers traditional Siamese backbones. It introduces M^2CD, a unified framework that combines modality-specific Mixture of Experts (MoE) with an Optical-to-SAR path (O2SP) and self-distillation to align features across modalities, compatible with both CNN and Transformer backbones. Key contributions include the MoE module for modality-aware representation learning and the O2SP-based training guidance that reduces cross-modal discrepancy without adding inference cost. Extensive experiments on the CAU-Flood data demonstrate state-of-the-art performance for the MiT-b1 variant, with ablations validating the effectiveness of both MoE and O2SP. The approach offers robust, efficient multimodal CD suitable for disaster response and other cross-modal remote sensing scenarios."

Abstract

Most existing change detection (CD) methods focus on optical images captured at different times, and deep learning (DL) has achieved remarkable success in this domain. However, in extreme scenarios such as disaster response, synthetic aperture radar (SAR), with its active imaging capability, is more suitable for providing post-event data. This introduces new challenges for CD methods, as existing weight-sharing Siamese networks struggle to effectively learn the cross-modal data distribution between optical and SAR images. To address this challenge, we propose a unified MultiModal CD framework, M$^2$CD. We integrate Mixture of Experts (MoE) modules into the backbone to explicitly handle diverse modalities, thereby enhancing the model's ability to learn multimodal data distributions. Additionally, we innovatively propose an Optical-to-SAR guided path (O2SP) and implement self-distillation during training to reduce the feature space discrepancy between different modalities, further alleviating the model's learning burden. We design multiple variants of M$^2$CD based on both CNN and Transformer backbones. Extensive experiments validate the effectiveness of the proposed framework, with the MiT-b1 version of M$^2$CD outperforming all state-of-the-art (SOTA) methods in optical-SAR CD tasks.

M$^2$CD: A Unified MultiModal Framework for Optical-SAR Change Detection with Mixture of Experts and Self-Distillation

TL;DR

This paper addresses optical-SAR change detection by bridging the modality gap that hampers traditional Siamese backbones. It introduces M^2CD, a unified framework that combines modality-specific Mixture of Experts (MoE) with an Optical-to-SAR path (O2SP) and self-distillation to align features across modalities, compatible with both CNN and Transformer backbones. Key contributions include the MoE module for modality-aware representation learning and the O2SP-based training guidance that reduces cross-modal discrepancy without adding inference cost. Extensive experiments on the CAU-Flood data demonstrate state-of-the-art performance for the MiT-b1 variant, with ablations validating the effectiveness of both MoE and O2SP. The approach offers robust, efficient multimodal CD suitable for disaster response and other cross-modal remote sensing scenarios."

Abstract

Most existing change detection (CD) methods focus on optical images captured at different times, and deep learning (DL) has achieved remarkable success in this domain. However, in extreme scenarios such as disaster response, synthetic aperture radar (SAR), with its active imaging capability, is more suitable for providing post-event data. This introduces new challenges for CD methods, as existing weight-sharing Siamese networks struggle to effectively learn the cross-modal data distribution between optical and SAR images. To address this challenge, we propose a unified MultiModal CD framework, MCD. We integrate Mixture of Experts (MoE) modules into the backbone to explicitly handle diverse modalities, thereby enhancing the model's ability to learn multimodal data distributions. Additionally, we innovatively propose an Optical-to-SAR guided path (O2SP) and implement self-distillation during training to reduce the feature space discrepancy between different modalities, further alleviating the model's learning burden. We design multiple variants of MCD based on both CNN and Transformer backbones. Extensive experiments validate the effectiveness of the proposed framework, with the MiT-b1 version of MCD outperforming all state-of-the-art (SOTA) methods in optical-SAR CD tasks.

Paper Structure

This paper contains 14 sections, 5 equations, 2 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. Proposed Method
  3. Overview of the M$^2$CD Framework
  4. MoE Module
  5. O2SP Self-Distillation
  6. Experiment
  7. Experimental Settings
  8. Dataset Description
  9. Comparative Methods
  10. Evaluation Metrics
  11. Implementation Details
  12. Experimental Results
  13. Ablation Study
  14. Conclusion

Figures (2)

  • Figure 1: Overview of the proposed M$^2$CD framework.
  • Figure 2: Visual comparison of different methods. For each set of images, from top-left to bottom-right: pre-event image, post-event image, ground truth, FC-EF, FC-Siam-Conc, FC-Siam-Diff, STANet, IFN, SNUNet, BIT, ChangeFormer-MiT-b0, ChangeFormer-MiT-b1, TinyCD, HANet, Changer-MiT-b0, Changer-MiT-b1, Changer-r18, Changer-s50, Changer-s101, CGNet, TTP, M$^2$CD-ConvNeXt-t, M$^2$CD-ConvNeXt-s, M$^2$CD-ConvNeXt-b, M$^2$CD-MiT-b0, and M$^2$CD-MiT-b1. Red indicates missed detections, and green indicates false alarms.