Deep Unfolding Network with Spatial Alignment for multi-modal MRI reconstruction

TL;DR

This work tackles slow MRI acquisition and cross-modality misalignment in multi-modal CS-MRI by introducing DUN-SA, a Deep Unfolding Network that embeds a spatial alignment task into the reconstruction process. It derives a joint alignment-reconstruction model with an aligned cross-modal prior term and solves it via alternating optimization, then unfolds the iterations into SAM (Spatial Alignment Module) and RM (Reconstruction Module) with AIPLB, ProxNet_Z, and ProxNet_S, enabling end-to-end training. The approach demonstrates superior reconstruction quality across four datasets (fastMRI, IXI, In-house, BraTS 2018) and shows robustness to varying misalignment and imperfect reference data, while preserving interpretability through stage-wise network modules. This has practical impact for faster, more accurate multi-modal MRI in clinical workflows, reducing the burden of undersampling artifacts while leveraging cross-modal information.

Abstract

Multi-modal Magnetic Resonance Imaging (MRI) offers complementary diagnostic information, but some modalities are limited by the long scanning time. To accelerate the whole acquisition process, MRI reconstruction of one modality from highly undersampled k-space data with another fully-sampled reference modality is an efficient solution. However, the misalignment between modalities, which is common in clinic practice, can negatively affect reconstruction quality. Existing deep learning-based methods that account for inter-modality misalignment perform better, but still share two main common limitations: (1) The spatial alignment task is not adaptively integrated with the reconstruction process, resulting in insufficient complementarity between the two tasks; (2) the entire framework has weak interpretability. In this paper, we construct a novel Deep Unfolding Network with Spatial Alignment, termed DUN-SA, to appropriately embed the spatial alignment task into the reconstruction process. Concretely, we derive a novel joint alignment-reconstruction model with a specially designed cross-modal spatial alignment term. By relaxing the model into cross-modal spatial alignment and multi-modal reconstruction tasks, we propose an effective algorithm to solve this model alternatively. Then, we unfold the iterative steps of the proposed algorithm and design corresponding network modules to build DUN-SA with interpretability. Through end-to-end training, we effectively compensate for spatial misalignment using only reconstruction loss, and utilize the progressively aligned reference modality to provide inter-modality prior to improve the reconstruction of the target modality. Comprehensive experiments on three real datasets demonstrate that our method exhibits superior reconstruction performance compared to state-of-the-art methods.

Figures (14)

• Figure 1: The overall structure of the proposed Deep Unfolding Network with Spatial Alignment (DUN-SA) consists of SAM (Spatial Alignment Module) and RM (Reconstruction Module). The RM is composed of AIPLB (Aligned Inter-modality Prior Learning Block), DB (Denoising Block), and DCB (Data Consistency Block). SAM is used to solve spatial alignment task, while RM is for reconstruction task. Specifically, AIPLB is used to learn aligned inter-modality prior, DB is used to learn denoising prior, and DCB is used to enforce data consistency constraint.
• Figure 2: Architecture of Spatial Alignment Module (SAM).
• Figure 3: Detailed configurations of SA-Net, ProxNet_Z and ProxNet_S.
• Figure 4: Visual comparison with representative methods for 4$\times$ acceleration under 1D equispaced subsampling mask on fastMRI dataset. First row: Reconstructed images by different methods; second row: Zoomed-in region of interest; third row: Equispaced mask of 4$\times$ acceleration and error maps of different methods.
• Figure 5: Visual comparison with representative methods for 8$\times$ acceleration under 1D equispaced subsampling mask on the IXI dataset. First row: Reconstructed images by different methods; second row: Zoomed-in region of interest; third row: Equispaced mask of 8$\times$ acceleration and error maps of different methods.