MFmamba: A Multi-function Network for Panchromatic Image Resolution Restoration Based on State-Space Model

TL;DR

The paper tackles the challenge of obtaining high-spatial-resolution color imagery from a single PAN input by unifying super-resolution and spectral recovery into a single network. MFmamba combines a UNet++ backbone with the Mamba Upsample Block (MUB) based on a state-space model, plus the Multi-scale Hybrid Cross Block (MHCB) and a Dual Pool Attention (DPA) module to enable SR, spectral recovery, and their joint task. Across five remote sensing datasets, MFmamba demonstrates competitive or superior performance in SR, spectral recovery, and joint recovery, while offering efficiency advantages over Transformer-based approaches. This work provides a practical workflow for generating HR color RS images from PAN data, with potential benefits for downstream interpretation and analysis in remote sensing applications.

Abstract

Remote sensing images are becoming increasingly widespread in military, earth resource exploration. Because of the limitation of a single sensor, we can obtain high spatial resolution grayscale panchromatic (PAN) images and low spatial resolution color multispectral (MS) images. Therefore, an important issue is to obtain a color image with high spatial resolution when there is only a PAN image at the input. The existing methods improve spatial resolution using super-resolution (SR) technology and spectral recovery using colorization technology. However, the SR technique cannot improve the spectral resolution, and the colorization technique cannot improve the spatial resolution. Moreover, the pansharpening method needs two registered inputs and can not achieve SR. As a result, an integrated approach is expected. To solve the above problems, we designed a novel multi-function model (MFmamba) to realize the tasks of SR, spectral recovery, joint SR and spectral recovery through three different inputs. Firstly, MFmamba utilizes UNet++ as the backbone, and a Mamba Upsample Block (MUB) is combined with UNet++. Secondly, a Dual Pool Attention (DPA) is designed to replace the skip connection in UNet++. Finally, a Multi-scale Hybrid Cross Block (MHCB) is proposed for initial feature extraction. Many experiments show that MFmamba is competitive in evaluation metrics and visual results and performs well in the three tasks when only the input PAN image is used.

Figures (9)

• Figure 1: General structure of the proposed solution (MFmamba).
• Figure 2: The structure of Multi-scale Hybrid Cross Block (MHCB).
• Figure 3: The structure of Dual Pool Attention (DPA).
• Figure 4: The structure of Mamba Upsample Module (MUB).
• Figure 5: Visual results of MFmamba and benchmark methods on five different datasets.