Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MCU-Net: A Multi-prior Collaborative Deep Unfolding Network with Gates-controlled Spatial Attention for Accelerated MR Image Reconstruction

Xiaoyu Qiao, Weisheng Li, Guofen Wang, Yuping Huang

TL;DR

This study designs a gates-controlled spatial attention module (GSAM), evaluating the relative confidence and overall confidence maps for intermediate reconstructions produced by different subnetworks, and designs correction modules to enhance the effectiveness in regions where both subnetworks exhibit limited performance.

Abstract

Deep unfolding networks (DUNs) have demonstrated significant potential in accelerating magnetic resonance imaging (MRI). However, they often encounter high computational costs and slow convergence rates. Besides, they struggle to fully exploit the complementarity when incorporating multiple priors. In this study, we propose a multi-prior collaborative DUN, termed MCU-Net, to address these limitations. Our method features a parallel structure consisting of different optimization-inspired subnetworks based on low-rank and sparsity, respectively. We design a gates-controlled spatial attention module (GSAM), evaluating the relative confidence (RC) and overall confidence (OC) maps for intermediate reconstructions produced by different subnetworks. RC allocates greater weights to the image regions where each subnetwork excels, enabling precise element-wise collaboration. We design correction modules to enhance the effectiveness in regions where both subnetworks exhibit limited performance, as indicated by low OC values, thereby obviating the need for additional branches. The gate units within GSAMs are designed to preserve necessary information across multiple iterations, improving the accuracy of the learned confidence maps and enhancing robustness against accumulated errors. Experimental results on multiple datasets show significant improvements on PSNR and SSIM results with relatively low FLOPs compared to cutting-edge methods. Additionally, the proposed strategy can be conveniently applied to various DUN structures to enhance their performance.

MCU-Net: A Multi-prior Collaborative Deep Unfolding Network with Gates-controlled Spatial Attention for Accelerated MR Image Reconstruction

TL;DR

This study designs a gates-controlled spatial attention module (GSAM), evaluating the relative confidence and overall confidence maps for intermediate reconstructions produced by different subnetworks, and designs correction modules to enhance the effectiveness in regions where both subnetworks exhibit limited performance.

Abstract

Deep unfolding networks (DUNs) have demonstrated significant potential in accelerating magnetic resonance imaging (MRI). However, they often encounter high computational costs and slow convergence rates. Besides, they struggle to fully exploit the complementarity when incorporating multiple priors. In this study, we propose a multi-prior collaborative DUN, termed MCU-Net, to address these limitations. Our method features a parallel structure consisting of different optimization-inspired subnetworks based on low-rank and sparsity, respectively. We design a gates-controlled spatial attention module (GSAM), evaluating the relative confidence (RC) and overall confidence (OC) maps for intermediate reconstructions produced by different subnetworks. RC allocates greater weights to the image regions where each subnetwork excels, enabling precise element-wise collaboration. We design correction modules to enhance the effectiveness in regions where both subnetworks exhibit limited performance, as indicated by low OC values, thereby obviating the need for additional branches. The gate units within GSAMs are designed to preserve necessary information across multiple iterations, improving the accuracy of the learned confidence maps and enhancing robustness against accumulated errors. Experimental results on multiple datasets show significant improvements on PSNR and SSIM results with relatively low FLOPs compared to cutting-edge methods. Additionally, the proposed strategy can be conveniently applied to various DUN structures to enhance their performance.
Paper Structure (20 sections, 29 equations, 8 figures, 8 tables, 1 algorithm)

This paper contains 20 sections, 29 equations, 8 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Single-prior-based DUNs
  4. Multi-prior-based DUNs
  5. Problem Formulation
  6. Method
  7. Network Structure
  8. Optimization-inspired Subnetworks
  9. Gates-controlled Spatial Attention
  10. Correction Modules
  11. Settings and Loss Function
  12. Experimental Results
  13. Datasets
  14. Comparison with cutting-edge methods
  15. Ablation studies
  16. ...and 5 more sections

Figures (8)

  • Figure 1: (a): the structure of MCU-Net. (b): the structure of the sparsity based sub-network ($N_s$ in (a)). (c): the structure of the low-rank based sub-network ($N_l$ in (a)). (d): the structure of GSAMs. (e): the structure of GCFFs in GSAMs. (f) the structure of CMs.
  • Figure 2: Examples of training data in the NYU dataset (the first row), FastMRI dataset (the second row) and the brain dataset (the third row). The first column displays the ground truth images, the second column shows the zero-filled images and the third column are the corresponding sampling masks.
  • Figure 3: Examples of reconstructed images of different methods. The first rows in each subfigures are the reconstructed images, the second rows are the zoomed details in the red boxes and the third rows are the corresponding error maps. Methods from left to right:1. ground truth; 2. zero-filled; 3. TV; 4. D5C5; 5. U-Net; 6. ISTA-Net; 7. VS-Net; 8. E2E-VarNet; 9. ReVarNet; 10. MeDL-Net; 11. vSharp; 12. MCU-Net (ours).
  • Figure 4: Examples of reconstructed brain MR images and the corresponding error maps. Methods from left to right: 1. ground truth; 2. zero-filled; 3. TV; 4. D5C5; 5. U-Net; 6. ISTA-Net; 7. VS-Net; 8. E2E-VarNet; 9. ReVarNet; 10. MeDL-Net; 11. vSharp; 12. MCU-Net (ours).
  • Figure 5: The test PSNR and SSIM results of L+S-Net and the proposed MCU-Net at 4-fold and 6-fold acceleration.
  • ...and 3 more figures