The R2D2 Deep Neural Network Series for Scalable Non-Cartesian Magnetic Resonance Imaging

TL;DR

This paper tackles scalable reconstruction for undersampled non-Cartesian MRI by introducing R2D2, a residual-to-residual DNN series that iteratively refines images using back-projected data residuals while keeping the measurement operator external to the network. The approach bypasses heavy NUFFT training costs and requires relatively few iterations, achieving superior quality versus unrolled nets and diffusion-based methods on simulated radial MRI data, with robust performance on real knee data. Key contributions include a sequentially trained DNN series (and an unrolled benchmark R2D2-Net), normalization strategies to stabilize training, and demonstration of scalability in both training and inference, particularly when using the U-WDSR backbone. The findings suggest R2D2 can deliver high-fidelity reconstructions at accelerated speeds, enabling practical deployment in large-scale and higher-dimensional non-Cartesian MRI applications, with broad implications for clinical workflows and research in accelerated MRI.

Abstract

We introduce the R2D2 Deep Neural Network (DNN) series paradigm for fast and scalable image reconstruction from highly-accelerated non-Cartesian k-space acquisitions in Magnetic Resonance Imaging (MRI). While unrolled DNN architectures provide a robust image formation approach via data-consistency layers, embedding non-uniform fast Fourier transform operators in a DNN can become impractical to train at large scale, e.g in 2D MRI with a large number of coils, or for higher-dimensional imaging. Plug-and-play approaches that alternate a learned denoiser blind to the measurement setting with a data-consistency step are not affected by this limitation but their highly iterative nature implies slow reconstruction. To address this scalability challenge, we leverage the R2D2 paradigm that was recently introduced to enable ultra-fast reconstruction for large-scale Fourier imaging in radio astronomy. R2D2's reconstruction is formed as a series of residual images iteratively estimated as outputs of DNN modules taking the previous iteration's data residual as input. The method can be interpreted as a learned version of the Matching Pursuit algorithm. A series of R2D2 DNN modules were sequentially trained in a supervised manner on the fastMRI dataset and validated for 2D multi-coil MRI in simulation and on real data, targeting highly under-sampled radial k-space sampling. Results suggest that a series with only few DNNs achieves superior reconstruction quality over its unrolled incarnation R2D2-Net (whose training is also much less scalable), and over the state-of-the-art diffusion-based "Decomposed Diffusion Sampler" approach (also characterised by a slower reconstruction process).

Figures (10)

• Figure 1: Illustration of the R2D2 algorithm. The image iterates $\bm{x}_{i-1}$ and corresponding back-projected data residuals $\bm{r}_{i-1}$ are provided as inputs to R2D2 DNN modules. The outputs of the R2D2 DNN modules are used to update the subsequent image iterates. The progression of image iterates and back-projected data residuals is represented by dashed red and green arrows respectively, while the sequence of learned residual images is represented by dashed blue arrows.
• Figure 2: Illustration of the U-WDSR model. (a) illustrates the U-WDSR architecture, and (b) depicts its WDSR layer. The WDSR residual body (in green boxes) is interlaced with the convolutional layers of the U-Net. WDSR consists of 16 consecutive residual blocks. At each stage, the spatial size of feature maps is indicated at the lower centre of each box. The number of channels is indicated at the outer edge of each box.
• Figure 3: Illustration of the non-Cartesian MRI problem. Panel (a) displays an example GT image. Panel (b) illustrates the radial sampling pattern in the spatial Fourier domain, superimposed to the Fourier transform of the GT image. Panel (c) shows the corresponding back-projected image.
• Figure 4: Quantitative reconstruction results from simulated experiments. The PSNR and SSIM of R2D2(U-Net) and R2D2(U-WDSR) (solid lines) are reported against the number of R2D2 iterations (panels (a) and (b); reported values are averages over 300 inverse problems across AFs), and the AF value (panels (c) and (d); reported values are averages over 50 inverse problems). Benchmark methods are represented via horizontal lines, with U-Net and U-WDSR represented by dotted lines, and R2D2-Net(U-Net), R2D2-Net(U-WDSR), NC-PDNet, and DDS represented by dashed lines.
• Figure 5: Visual reconstruction results for one of test inverse problems of the simulated experiments, with $\mathrm{AF=16}$ and 16 coils (flip with Figure \ref{['fig:simulation_4']} for $\mathrm{AF=4}$ comparison at a glance). The first and fourth rows show the GT image and estimated images of different methods. The second and fifth rows show the selected ROI marked by boxes in the images. Dotted ellipses in these ROIs highlight key areas with subtle differences between the estimated images. The third and sixth rows present the error images between the GT and estimated images. Values of PSNR (dB) are reported in the bottom left of the estimated images.