A Luminance-Aware Multi-Scale Network for Polarization Image Fusion with a Multi-Scene Dataset
Zhuangfan Huang, Xiaosong Li, Gao Wang, Tao Ye, Haishu Tan, Huafeng Li
TL;DR
This work tackles polarization image fusion under complex lighting by introducing MLSN, a luminance-aware, multi-scale network that fuses $S_{0}$ and $DOLP$ with a Brightness-Branch and a Brightness-Enhancement module, guided by windowed self-attention in a Swin-Transformer bottleneck. A multi-objective loss $L_{all}$ harmonizes structural fidelity, pixel accuracy, texture, and contrast to produce high-quality fused images. The authors also present MSP, a large multi-scene polarization dataset with 1000 pairs and four-direction polarization mappings to advance data-driven polarization fusion research. Experimental results on MSP, PIF, and GAND show state-of-the-art performance in both subjective and objective metrics, with strong ablation evidence for the luminance-guided components. The work offers practical impact for surveillance, autonomous systems, and material characterization in challenging lighting conditions.
Abstract
Polarization image fusion combines S0 and DOLP images to reveal surface roughness and material properties through complementary texture features, which has important applications in camouflage recognition, tissue pathology analysis, surface defect detection and other fields. To intergrate coL-Splementary information from different polarized images in complex luminance environment, we propose a luminance-aware multi-scale network (MLSN). In the encoder stage, we propose a multi-scale spatial weight matrix through a brightness-branch , which dynamically weighted inject the luminance into the feature maps, solving the problem of inherent contrast difference in polarized images. The global-local feature fusion mechanism is designed at the bottleneck layer to perform windowed self-attention computation, to balance the global context and local details through residual linking in the feature dimension restructuring stage. In the decoder stage, to further improve the adaptability to complex lighting, we propose a Brightness-Enhancement module, establishing the mapping relationship between luminance distribution and texture features, realizing the nonlinear luminance correction of the fusion result. We also present MSP, an 1000 pairs of polarized images that covers 17 types of indoor and outdoor complex lighting scenes. MSP provides four-direction polarization raw maps, solving the scarcity of high-quality datasets in polarization image fusion. Extensive experiment on MSP, PIF and GAND datasets verify that the proposed MLSN outperms the state-of-the-art methods in subjective and objective evaluations, and the MS-SSIM and SD metircs are higher than the average values of other methods by 8.57%, 60.64%, 10.26%, 63.53%, 22.21%, and 54.31%, respectively. The source code and dataset is avalable at https://github.com/1hzf/MLS-UNet.