Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning

Qi Chen, Xiaohan Xing, Zhen Chen, Zhiwei Xiong

TL;DR

FSMNet addresses the challenge of accelerating multi-contrast MR reconstruction by modeling cross-modal dependencies with a frequency-spatial mutual learning framework. It introduces the Frequency-Spatial Feature Extraction (FSFE) to obtain global Fourier-domain features and local spatial features, a Cross-Modal Selective fusion (CMS-fusion) to selectively incorporate auxiliary-modality information, and a Frequency-Spatial fusion (FS-fusion) to jointly integrate features. The method achieves state-of-the-art results on BraTS and fastMRI across $4\times$ and $8\times$ acceleration, validated by ablations showing the contributions of FSFE, CMS-fusion, and FS-fusion. The work provides a computationally efficient path to high-quality MCMR in clinical settings and releases code for replication.

Abstract

To accelerate Magnetic Resonance (MR) imaging procedures, Multi-Contrast MR Reconstruction (MCMR) has become a prevalent trend that utilizes an easily obtainable modality as an auxiliary to support high-quality reconstruction of the target modality with under-sampled k-space measurements. The exploration of global dependency and complementary information across different modalities is essential for MCMR. However, existing methods either struggle to capture global dependency due to the limited receptive field or suffer from quadratic computational complexity. To tackle this dilemma, we propose a novel Frequency and Spatial Mutual Learning Network (FSMNet), which efficiently explores global dependencies across different modalities. Specifically, the features for each modality are extracted by the Frequency-Spatial Feature Extraction (FSFE) module, featuring a frequency branch and a spatial branch. Benefiting from the global property of the Fourier transform, the frequency branch can efficiently capture global dependency with an image-size receptive field, while the spatial branch can extract local features. To exploit complementary information from the auxiliary modality, we propose a Cross-Modal Selective fusion (CMS-fusion) module that selectively incorporate the frequency and spatial features from the auxiliary modality to enhance the corresponding branch of the target modality. To further integrate the enhanced global features from the frequency branch and the enhanced local features from the spatial branch, we develop a Frequency-Spatial fusion (FS-fusion) module, resulting in a comprehensive feature representation for the target modality. Extensive experiments on the BraTS and fastMRI datasets demonstrate that the proposed FSMNet achieves state-of-the-art performance for the MCMR task with different acceleration factors. The code is available at: https://github.com/qic999/FSMNet.

Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning

TL;DR

FSMNet addresses the challenge of accelerating multi-contrast MR reconstruction by modeling cross-modal dependencies with a frequency-spatial mutual learning framework. It introduces the Frequency-Spatial Feature Extraction (FSFE) to obtain global Fourier-domain features and local spatial features, a Cross-Modal Selective fusion (CMS-fusion) to selectively incorporate auxiliary-modality information, and a Frequency-Spatial fusion (FS-fusion) to jointly integrate features. The method achieves state-of-the-art results on BraTS and fastMRI across and acceleration, validated by ablations showing the contributions of FSFE, CMS-fusion, and FS-fusion. The work provides a computationally efficient path to high-quality MCMR in clinical settings and releases code for replication.

Abstract

To accelerate Magnetic Resonance (MR) imaging procedures, Multi-Contrast MR Reconstruction (MCMR) has become a prevalent trend that utilizes an easily obtainable modality as an auxiliary to support high-quality reconstruction of the target modality with under-sampled k-space measurements. The exploration of global dependency and complementary information across different modalities is essential for MCMR. However, existing methods either struggle to capture global dependency due to the limited receptive field or suffer from quadratic computational complexity. To tackle this dilemma, we propose a novel Frequency and Spatial Mutual Learning Network (FSMNet), which efficiently explores global dependencies across different modalities. Specifically, the features for each modality are extracted by the Frequency-Spatial Feature Extraction (FSFE) module, featuring a frequency branch and a spatial branch. Benefiting from the global property of the Fourier transform, the frequency branch can efficiently capture global dependency with an image-size receptive field, while the spatial branch can extract local features. To exploit complementary information from the auxiliary modality, we propose a Cross-Modal Selective fusion (CMS-fusion) module that selectively incorporate the frequency and spatial features from the auxiliary modality to enhance the corresponding branch of the target modality. To further integrate the enhanced global features from the frequency branch and the enhanced local features from the spatial branch, we develop a Frequency-Spatial fusion (FS-fusion) module, resulting in a comprehensive feature representation for the target modality. Extensive experiments on the BraTS and fastMRI datasets demonstrate that the proposed FSMNet achieves state-of-the-art performance for the MCMR task with different acceleration factors. The code is available at: https://github.com/qic999/FSMNet.
Paper Structure (10 sections, 8 equations, 2 figures, 2 tables)

This paper contains 10 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Frequency-Spatial Feature Extraction
  4. Cross-Modal Frequency-Spatial Feature Fusion
  5. Loss Function
  6. Experiments
  7. Datasets and Implementation Details
  8. Experimental Results
  9. Conclusion
  10. Disclosure of Interests.

Figures (2)

  • Figure 1: Overview of FSMNet: In each stage, the FSFE module extracts global and local features from the frequency and spatial branches, respectively. The CMS-fusion module integrates the multi-modal features for each branch, and subsequently, the FS-fusion module combines the features across the frequency and spatial branches.
  • Figure 2: Qualitative visualization of the reconstructed images (1st row) and error maps (2nd row) for different MCMR methods with $4\times$ AF on the BraTS dataset. Additional qualitative results are provided in the supplementary material.