Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

First-order State Space Model for Lightweight Image Super-resolution

Yujie Zhu, Xinyi Zhang, Yekai Lu, Guang Yang, Faming Fang, Guixu Zhang

TL;DR

This paper tackles lightweight image super-resolution by enhancing a Mamba-based SSM backbone with a First-order State Space Model (FSSM). By deriving a discrete, first-order hold-based formulation and introducing higher-order approximations, the authors enable stronger token correlations without increasing parameters, and provide a cumulative error analysis showing favorable bounds relative to vanilla SSM. Extensive experiments on DIV2K/DF2K and five benchmark datasets demonstrate that FMambaIR with FSSM variants achieves state-of-the-art results among light-weight SR methods, improving PSNR/SSIM and visual quality. The work offers a practical, efficient backbone for vision tasks requiring long-range dependencies and paves the way for broader adoption of FSSM in related domains.

Abstract

State space models (SSMs), particularly Mamba, have shown promise in NLP tasks and are increasingly applied to vision tasks. However, most Mamba-based vision models focus on network architecture and scan paths, with little attention to the SSM module. In order to explore the potential of SSMs, we modified the calculation process of SSM without increasing the number of parameters to improve the performance on lightweight super-resolution tasks. In this paper, we introduce the First-order State Space Model (FSSM) to improve the original Mamba module, enhancing performance by incorporating token correlations. We apply a first-order hold condition in SSMs, derive the new discretized form, and analyzed cumulative error. Extensive experimental results demonstrate that FSSM improves the performance of MambaIR on five benchmark datasets without additionally increasing the number of parameters, and surpasses current lightweight SR methods, achieving state-of-the-art results.

First-order State Space Model for Lightweight Image Super-resolution

TL;DR

This paper tackles lightweight image super-resolution by enhancing a Mamba-based SSM backbone with a First-order State Space Model (FSSM). By deriving a discrete, first-order hold-based formulation and introducing higher-order approximations, the authors enable stronger token correlations without increasing parameters, and provide a cumulative error analysis showing favorable bounds relative to vanilla SSM. Extensive experiments on DIV2K/DF2K and five benchmark datasets demonstrate that FMambaIR with FSSM variants achieves state-of-the-art results among light-weight SR methods, improving PSNR/SSIM and visual quality. The work offers a practical, efficient backbone for vision tasks requiring long-range dependencies and paves the way for broader adoption of FSSM in related domains.

Abstract

State space models (SSMs), particularly Mamba, have shown promise in NLP tasks and are increasingly applied to vision tasks. However, most Mamba-based vision models focus on network architecture and scan paths, with little attention to the SSM module. In order to explore the potential of SSMs, we modified the calculation process of SSM without increasing the number of parameters to improve the performance on lightweight super-resolution tasks. In this paper, we introduce the First-order State Space Model (FSSM) to improve the original Mamba module, enhancing performance by incorporating token correlations. We apply a first-order hold condition in SSMs, derive the new discretized form, and analyzed cumulative error. Extensive experimental results demonstrate that FSSM improves the performance of MambaIR on five benchmark datasets without additionally increasing the number of parameters, and surpasses current lightweight SR methods, achieving state-of-the-art results.

Paper Structure

This paper contains 23 sections, 28 equations, 5 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Image Super-resolution
  4. State Space Model
  5. Method
  6. Overview of 2D First-order Selective Scan Module
  7. First-order State Space Model (FSSM)
  8. Approximation
  9. Analysis
  10. Theorem 1
  11. Theorem 2
  12. Experiments
  13. Datasets and Details
  14. Datasets and Evaluation
  15. Training Details
  16. ...and 8 more sections

Figures (5)

  • Figure 1: Trade-off between the number of model parameters and performance on Urban100(x4).
  • Figure 2: The architecture diagram of 2D First-order Selective Scan Module.
  • Figure 3: SSM with different extensions.
  • Figure 4: Components of cumulative error.
  • Figure 5: Qualitative Comparison of our FMambaIR model with SOTA lightweight SR models on x4 upscaling task.