Performance of a GPU- and Time-Efficient Pseudo 3D Network for Magnetic Resonance Image Super-Resolution and Motion Artifact Reduction

TL;DR

A unified 2D deep learning framework for pseudo-3D MRI image super-resolution reconstruction (SRR) and motion artifact reduction (MAR) and TS-RCAN, which outperformed the 3D networks of mDCSRN and ReCNN and has potential use for SRR and MAR in the clinical setting.

Abstract

Shortening acquisition time and reducing motion artifacts are the most critical challenges in magnetic resonance imaging (MRI). Deep learning-based image restoration has emerged as a promising solution capable of generating high-resolution and motion-artifact-free MRI images from low-resolution images acquired with shortened acquisition times or from motion-artifact-corrupted images. To facilitate clinical integration, a time- and GPU-efficient network with reliable accuracy is essential. In this study, we adopted a unified 2D deep learning framework for pseudo-3D MRI image super-resolution reconstruction (SRR) and motion artifact reduction (MAR). The optimal down-sampling factors to optimize the acquisition time in SRR were identified. Training for MAR was performed using publicly available in vivo data, employing a novel standardized method to induce motion artifacts of varying severity in a controlled way. The accuracy of the network was evaluated through a pixel-wise uncertainty map, and performance was benchmarked against state-of-the-art methods. The results demonstrated that the down-sampling factor of 1x1x2 for x2 acceleration and 2x2x2 for x4 acceleration was optimal. For SRR, the proposed TS-RCAN outperformed the 3D networks of mDCSRN and ReCNN, with an improvement of more than 0.01 in SSIM and 1.5 dB in PSNR while reducing GPU load by up to and inference time by up to 90%. For MAR, TS-RCAN exceeded UNet's performance by up to 0.014 in SSIM and 1.48 dB in PSNR. Additionally, TS-RCAN provided uncertainty information, which can be used to estimate the quality of the reconstructed images. TS-RCAN has potential use for SRR and MAR in the clinical setting.

Figures (9)

• Figure 1: Pipeline of thin-slab RCAN (TS-RCAN) used for MRI super resolution reconstruction and motion artifact reduction.
• Figure 2: Retrospective generation of 3D low resolution and MA-corrupted MR images. (A): Generation of 3D low resolution images with scale factor of $2\times2\times2$ and patch cropping. (B): Image-based generation of MA in MR images. (C): Scheme of motion pattern employed in our study.
• Figure 3: Dependency of SRR performance on down-sampling factors. The scale factors are grouped according to the acceleration factor. The $1\times1\times2$ and $2\times2\times2$ down-sampling with $M\ge3$ achieved the highest mean values of SSIM/PSNR values for $\times2$ and $\times4$ acceleration respectively. With $\times2$ acceleration, $1\times1\times2$ down-sampling with $M=5$ and self-ensemble significantly outperformed all cases of $2\times2\times1$ in PSNR and SSIM. With $\times4$ acceleration, $2\times2\times2$ down-sampling with $M\ge3$ and self-ensemble significantly outperformed $4\times4\times1$ in SSIM. (A)/(C): PSNR and SSIM of super-resolution images with $\times2$ acceleration ($2\times2\times1$ and $1\times1\times2$); (B)/(D): PSNR and SSIM of super-resolution images with $\times4$ acceleration ($4\times4\times1$, $1\times1\times4$ and $2\times2\times2$). ‘+’ represents the SRR with self-ensemble. Significant difference is indicated with $*p<0.05$ and $**p<0.0005$.
• Figure 4: Qualitative comparison of 3D SRR with different down-sampling factors of $\times4$ acceleration. Note the loss of anatomical details in the cerebellar grey-white-matter differentiability in the sagittal image (arrow) of the $4\times4\times1$ down-sampling strategy, as well as the loss of one/both anterior cerebral arteries (arrows) in the axial images of the $4\times4\times1$ and $1\times1\times4$ down-sampling factor. Best qualitative results were achieved using the $2\times2\times2$ down-sampling factor.
• Figure 5: Comparison of TS-RCAN with other state-of-the-art 3D networks. (A)-(D): comparison of metrics with two scale factors ($2\times2\times1$ and $2\times2\times2$), ‘+’ represents the SRR with self-ensemble. TS-RCAN achieved comparable performance with MINet and outperformed the other networks in SSIM and PSNR. Significant difference is indicated with $*p<0.05$ and $**p<0.0005$. (E)-(J): Comparison of the number of operations, GPU consumption and inference time to PSNR and SSIM with scale factor of $2\times2\times2$. TS-RCAN achieved top performance by consuming the minimal computation resources and inference time.