$\textrm{A}^{\textrm{2}}$RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion

TL;DR

This work tackles the vulnerability of infrared-visible image fusion (IVIF) to adversarial perturbations by introducing the adversarial attack resilient network, $A^{2}$RNet, and a novel anti-attack loss $\mathcal{L}_{a}$. The method combines a Unet-based fusion backbone with a transformer-based Defensive Refinement Module (DRM) and Adversarial-Resistant Blocks (ARBs) that employ Mercer-based self-attention to maintain robust feature fusion under attack; adversarial training uses PGD-generated examples with pseudo-labels to guide the min-max optimization of $\mathcal{L}_{a}$. Experimental results on MFNet and $M^{3}$FD demonstrate improved fusion quality and stronger resilience in downstream tasks (detection and segmentation) under adversarial perturbations, outperforming seven SOTA methods. The work provides a practical framework for robust IVIF with potential for broader application in safety-critical vision tasks and sets the ground for future data-centric robustness in fusion systems.

Abstract

Infrared and visible image fusion (IVIF) is a crucial technique for enhancing visual performance by integrating unique information from different modalities into one fused image. Exiting methods pay more attention to conducting fusion with undisturbed data, while overlooking the impact of deliberate interference on the effectiveness of fusion results. To investigate the robustness of fusion models, in this paper, we propose a novel adversarial attack resilient network, called $\textrm{A}^{\textrm{2}}$RNet. Specifically, we develop an adversarial paradigm with an anti-attack loss function to implement adversarial attacks and training. It is constructed based on the intrinsic nature of IVIF and provide a robust foundation for future research advancements. We adopt a Unet as the pipeline with a transformer-based defensive refinement module (DRM) under this paradigm, which guarantees fused image quality in a robust coarse-to-fine manner. Compared to previous works, our method mitigates the adverse effects of adversarial perturbations, consistently maintaining high-fidelity fusion results. Furthermore, the performance of downstream tasks can also be well maintained under adversarial attacks. Code is available at https://github.com/lok-18/A2RNet.

Figures (8)

• Figure 1: Schematic illustration of different adversarial operations. Clearly, fused images generated by attacked image pairs exhibit superior qualitative, quantitative and downstream task performance when conducting the proposed framework.
• Figure 2: Framework of the proposed $\textrm{A}^{\textrm{2}}$RNet. In specific, (a) and (b) represent the adversarial examples generation and adversarial training processing, respectively. (c) is the adversarial attack resilient network, which contains the defensive refinement module (DRM) as shown in (d).
• Figure 3: Fusion comparisons with SOTA methods in MFNet and $\textrm{M}^{3}$FD datasets. We apply PGD to clean samples and add perturbations with $\epsilon = 4/255$ to generate AEs. The signal maps are also provided for clean and attack states. The closer the waveform, the stronger the robustness.
• Figure 4: Bar charts of the fusion comparison metrics. For better visualization, we have scaled the values of certain metrics.
• Figure 5: Detection and segmentation comparisons of fused images. Under adversarial conditions, our method yields better performance in downstream tasks.