Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation

Jiahao Huang, Liutao Yang, Fanwen Wang, Yang Nan, Angelica I. Aviles-Rivero, Carola-Bibiane Schönlieb, Daoqiang Zhang, Guang Yang

TL;DR

This study introduces MambaMIR, a Mamba-based model for medical image reconstruction, as well as its Generative Adversarial Network-based variant, MambaMIR-GAN, which inherits several advantages, such as linear complexity, global receptive fields, and dynamic weights, from the original Mamba model.

Abstract

The recent Mamba model has shown remarkable adaptability for visual representation learning, including in medical imaging tasks. This study introduces MambaMIR, a Mamba-based model for medical image reconstruction, as well as its Generative Adversarial Network-based variant, MambaMIR-GAN. Our proposed MambaMIR inherits several advantages, such as linear complexity, global receptive fields, and dynamic weights, from the original Mamba model. The innovated arbitrary-mask mechanism effectively adapt Mamba to our image reconstruction task, providing randomness for subsequent Monte Carlo-based uncertainty estimation. Experiments conducted on various medical image reconstruction tasks, including fast MRI and SVCT, which cover anatomical regions such as the knee, chest, and abdomen, have demonstrated that MambaMIR and MambaMIR-GAN achieve comparable or superior reconstruction results relative to state-of-the-art methods. Additionally, the estimated uncertainty maps offer further insights into the reliability of the reconstruction quality. The code is publicly available at https://github.com/ayanglab/MambaMIR.

MambaMIR: An Arbitrary-Masked Mamba for Joint Medical Image Reconstruction and Uncertainty Estimation

TL;DR

This study introduces MambaMIR, a Mamba-based model for medical image reconstruction, as well as its Generative Adversarial Network-based variant, MambaMIR-GAN, which inherits several advantages, such as linear complexity, global receptive fields, and dynamic weights, from the original Mamba model.

Abstract

The recent Mamba model has shown remarkable adaptability for visual representation learning, including in medical imaging tasks. This study introduces MambaMIR, a Mamba-based model for medical image reconstruction, as well as its Generative Adversarial Network-based variant, MambaMIR-GAN. Our proposed MambaMIR inherits several advantages, such as linear complexity, global receptive fields, and dynamic weights, from the original Mamba model. The innovated arbitrary-mask mechanism effectively adapt Mamba to our image reconstruction task, providing randomness for subsequent Monte Carlo-based uncertainty estimation. Experiments conducted on various medical image reconstruction tasks, including fast MRI and SVCT, which cover anatomical regions such as the knee, chest, and abdomen, have demonstrated that MambaMIR and MambaMIR-GAN achieve comparable or superior reconstruction results relative to state-of-the-art methods. Additionally, the estimated uncertainty maps offer further insights into the reliability of the reconstruction quality. The code is publicly available at https://github.com/ayanglab/MambaMIR.
Paper Structure (18 sections, 7 equations, 7 figures)

This paper contains 18 sections, 7 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Preliminaries
  4. Medical Image Reconstruction.
  5. State Space Model and its Discretisation.
  6. Vision Mamba.
  7. Arbitrary-Masked S6 Block with Monte Carlo Dropout
  8. MambaMIR: Model Architecture
  9. Overall Architecture.
  10. AMSS Block.
  11. Optimisation Scheme
  12. Experimental Results
  13. Dataset
  14. FastMRI.
  15. Low-Dose CT Image and Projection Datasets.
  16. ...and 3 more sections

Figures (7)

  • Figure 1: The detailed structure of the proposed Arbitrary-Masked S6 (AMS6) Block.
  • Figure 2: (A) The architecture of the proposed MambaMIR. The mamba is composed of are $m$ Arbitrary-Masked State Space (AMSS) Block Groups, where each group contains $n$ AMSS Block; (B) The structure of the AMSS Block; (C) The structure of the Arbitrary-Masked S6 (AMS6) Block.
  • Figure 3: The visualised results of comparison experiments on FastMRI dataset with acceleration factor (AF) $\times 8$.
  • Figure 4: The visualised results of comparison experiments on FastMRI dataset with acceleration factor (AF) $\times 16$.
  • Figure 5: The visualised results of comparison experiments on the abdomen subset of Low-Dose CT Image and Projection datasets.
  • ...and 2 more figures