QMambaBSR: Burst Image Super-Resolution with Query State Space Model

TL;DR

The paper addresses burst image super-resolution by modeling inter-frame and intra-frame sub-pixel information using a novel Query State Space Model (QSSM) that enables the base frame to query multiple current frames while suppressing noise. It introduces an Adaptive Up-sampling module (AdaUp) to adapt the upsampling kernel according to the spatial distribution of sub-pixel information, and a Multi-scale Fusion Module (MSFM) to fuse sub-pixel cues across scales. Together, these components yield state-of-the-art results on synthetic and real BurstSR benchmarks, validated by quantitative gains, ablations, and a user study. The approach offers practical gains in texture detail and artifact reduction in real-world burst images, with potential applicability to other burst restoration tasks.

Abstract

Burst super-resolution aims to reconstruct high-resolution images with higher quality and richer details by fusing the sub-pixel information from multiple burst low-resolution frames. In BusrtSR, the key challenge lies in extracting the base frame's content complementary sub-pixel details while simultaneously suppressing high-frequency noise disturbance. Existing methods attempt to extract sub-pixels by modeling inter-frame relationships frame by frame while overlooking the mutual correlations among multi-current frames and neglecting the intra-frame interactions, leading to inaccurate and noisy sub-pixels for base frame super-resolution. Further, existing methods mainly employ static upsampling with fixed parameters to improve spatial resolution for all scenes, failing to perceive the sub-pixel distribution difference across multiple frames and cannot balance the fusion weights of different frames, resulting in over-smoothed details and artifacts. To address these limitations, we introduce a novel Query Mamba Burst Super-Resolution (QMambaBSR) network, which incorporates a Query State Space Model (QSSM) and Adaptive Up-sampling module (AdaUp). Specifically, based on the observation that sub-pixels have consistent spatial distribution while random noise is inconsistently distributed, a novel QSSM is proposed to efficiently extract sub-pixels through inter-frame querying and intra-frame scanning while mitigating noise interference in a single step. Moreover, AdaUp is designed to dynamically adjust the upsampling kernel based on the spatial distribution of multi-frame sub-pixel information in the different burst scenes, thereby facilitating the reconstruction of the spatial arrangement of high-resolution details. Extensive experiments on four popular synthetic and real-world benchmarks demonstrate that our method achieves a new state-of-the-art performance.

Figures (6)

• Figure 1: The concat and frame-by-frame operations in existing methods struggle to efficiently extract sub-pixels and suppress noise, leading to remaining artifacts and over-smoothed details, as shown in (a). We observe that noise randomly appears on several frames, while effective sub-pixels have consistent intensity at corresponding positions in all frames, as shown in (c). Based on this, a novel inter-frame query and intra-frame scanning-based QMambaBSR is proposed to extract more accurate sub-pixels while mitigating noise interference simultaneously, as shown in (b).
• Figure 2: The overall framework of our proposed QMambaBSR, primarily includes the novel Query State Space Model (QSSM), Multi-scale Fusion Module (MSFM), and the Adaptive Up-sampling Module (AdaUp).
• Figure 3: Illustrative comparison between the proposed QSSM and existing cross-attention mechanisms.
• Figure 4: Visual comparison results with different methods on SyntheticBurst datasets for ×4 BurstSR.
• Figure 5: Visual comparison results with different methods on RealBSR-RGB dataset for ×4 BurstSR.