Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification

TL;DR

This work tackles three core weaknesses of Vision Transformers in MRI reconstruction under undersampling: loss of high-frequency details, noisy interactions among unrelated tokens, and poor multi-scale feature modeling. It introduces FPS-Former, a ViT-based encoder-decoder that integrates Frequency Modulation Attention Module (FMAM) to preserve high-frequency content, Spatial Purification Attention Module (SPAM) to restrict self-attention to content-related tokens, and Scale Diversification Feed-forward Network (SDFN) to model multi-scale information, complemented by Hybrid Experts Feature Refinement (HEFR) for feature refinement. Across CC359, fastMRI, and SKM-TEA, FPS-Former achieves state-of-the-art reconstruction performance with lower computational costs on single- and multi-coil data and under various undersampling masks. The results demonstrate substantial improvements in PSNR and SSIM, enhanced edge preservation, and robust generalization, signaling practical impact for faster, more reliable MRI scans. The approach advances ViT-based medical image reconstruction by combining frequency-aware attention, content-aware token grouping, and multi-scale feature fusion into a cohesive, efficient framework.

Abstract

The accelerated MRI reconstruction process presents a challenging ill-posed inverse problem due to the extensive under-sampling in k-space. Recently, Vision Transformers (ViTs) have become the mainstream for this task, demonstrating substantial performance improvements. However, there are still three significant issues remain unaddressed: (1) ViTs struggle to capture high-frequency components of images, limiting their ability to detect local textures and edge information, thereby impeding MRI restoration; (2) Previous methods calculate multi-head self-attention (MSA) among both related and unrelated tokens in content, introducing noise and significantly increasing computational burden; (3) The naive feed-forward network in ViTs cannot model the multi-scale information that is important for image restoration. In this paper, we propose FPS-Former, a powerful ViT-based framework, to address these issues from the perspectives of frequency modulation, spatial purification, and scale diversification. Specifically, for issue (1), we introduce a frequency modulation attention module to enhance the self-attention map by adaptively re-calibrating the frequency information in a Laplacian pyramid. For issue (2), we customize a spatial purification attention module to capture interactions among closely related tokens, thereby reducing redundant or irrelevant feature representations. For issue (3), we propose an efficient feed-forward network based on a hybrid-scale fusion strategy. Comprehensive experiments conducted on three public datasets show that our FPS-Former outperforms state-of-the-art methods while requiring lower computational costs.

Figures (4)

• Figure 1: Our main idea. (a) The pipeline of standard ViTs block. (b) ViTs suffer limitations of high-frequency attenuation, irrelevant token interactions, and a lack of multi-scale feature representation. (3) We propose to tackle the above issues from the perspectives of frequency modulation, spatial purification, and scale diversification, thereby enhancing the performance of ViT-based MRI reconstruction.
• Figure 2: (a) The overall architecture of the proposed FPS-Former. Given an input image $I_{in}$, we first apply a $3 \times 3$ convolution to obtain patch tokens. In the network backbone, we stack multiple FPS blocks to extract hierarchical features. FPS, consisting of FMAM (b), SPAM (c), and SDFN (d), is designed to tackle the issues of ViT-based MRI reconstruction. Besides, at both early and final stages of the network, we design HEFR (e) to provide refined features, ensuring the reconstruction of high-quality output $I_{out}$. (f) The motivation of our SPAM. MRI images contain widely distributed, similar patches that appear in groups.
• Figure 3: Qualitative comparison of different methods on (a) the single-coil dataset including CC359 and fastMRI, (b) the multi-coil dataset SKM-TEA, and (c) the CC359 dataset using different undersampling masks. The second row of each subplot shows the corresponding error maps. The red boxes and yellow ellipses highlight the details in the reconstruction results.
• Figure 4: (a) Frequency response analysis. (b) Visualization results of high-frequency details in reconstructed images.