M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for Optical-SAR Fusion Object Detection
Chao Wang, Wei Lu, Xiang Li, Jian Yang, Lei Luo
TL;DR
The paper tackles the challenge of robust object detection in remote sensing by (1) constructing M4-SAR, the first large-scale, precisely aligned optical–SAR fusion dataset with 112,184 image pairs, 981,862 instances across six categories, and multi-resolution/multi-scene coverage; (2) providing MSRODet, a standardized benchmarking toolkit that integrates major fusion methods for fair comparison; and (3) introducing E2E-OSDet, an end-to-end fusion framework with dedicated modules (Filter Augment Module, Cross-modal Mamba Interaction Module, Area-Attention Fusion Module) to bridge cross-modal gaps. Empirical results show multi-source fusion outperforms single-source baselines, with E2E-OSDet achieving a top $mAP$ of 61.4% and favorable inference efficiency, underscoring the value of optical–SAR synergy in challenging environments. The work also offers a practical path for future research by delivering a public dataset and codebase (MSRODet) to enable reproducible cross-modal detection studies. Overall, M4-SAR, MSRODet, and E2E-OSDet collectively advance robust, scalable multi-source remote sensing perception for real-world applications.
Abstract
Single-source remote sensing object detection using optical or SAR images struggles in complex environments. Optical images offer rich textural details but are often affected by low-light, cloud-obscured, or low-resolution conditions, reducing the detection performance. SAR images are robust to weather, but suffer from speckle noise and limited semantic expressiveness. Optical and SAR images provide complementary advantages, and fusing them can significantly improve the detection accuracy. However, progress in this field is hindered by the lack of large-scale, standardized datasets. To address these challenges, we propose the first comprehensive dataset for optical-SAR fusion object detection, named Multi-resolution, Multi-polarization, Multi-scene, Multi-source SAR dataset (M4-SAR). It contains 112,184 precisely aligned image pairs and nearly one million labeled instances with arbitrary orientations, spanning six key categories. To enable standardized evaluation, we develop a unified benchmarking toolkit that integrates six state-of-the-art multi-source fusion methods. Furthermore, we propose E2E-OSDet, a novel end-to-end multi-source fusion detection framework that mitigates cross-domain discrepancies and establishes a robust baseline for future studies. Extensive experiments on M4-SAR demonstrate that fusing optical and SAR data can improve $mAP$ by 5.7\% over single-source inputs, with particularly significant gains in complex environments. The dataset and code are publicly available at https://github.com/wchao0601/M4-SAR.