Evaluating the resolution of AI-based accelerated MR reconstruction using a deep learning-based model observer

TL;DR

The results demonstrate that AI-based accelerated MR reconstruction may produce visually pleasing images but may not achieve performance comparable to that of rSOS 1x, and the proposed DLMO approach may be employed to characterize the discriminative efficacy of AI-based undersampled reconstruction in MRI.

Abstract

We developed a deep learning-based model observer (DLMO) to evaluate a multi-coil sensitivity encoding parallel MRI system at different accelerations on the Rayleigh discrimination task as a surrogate measure of resolution. We inserted Gaussian-convolved doublet and singlet signals into the white matter area of synthetic brain images. K-space raw data were acquired by using a simulated MR imaging system at acceleration factors of one (fully sampled), four and eight. These raw data were reconstructed using a conventional root-sum-of-squares (rSOS) method and an U-Net method. DLMOs were first trained with fully sampled images and then re-trained for each acceleration using a transfer learning approach. These DLMOs had a similar discrimination performance as trained human readers, using a human-label alignment training strategy. The resolution of rSOS- and U-Net-reconstructed images was assessed using the area under the receiver operating characteristic curve (AUC). We observed that the U-Net method yielded significantly higher PSNR and SSIM than rSOS across different accelerations. However, task-based evaluation using the proposed DLMO revealed that the U-Net underperformed relative to the fully sampled reconstruction (i.e. rSOS 1x). Although U-Net at an acceleration factor of four exhibited modest gains over rSOS at the same acceleration for short signals, its AUC decreased by approximately 25% and 5% for 4 mm and 5 mm signals, respectively, compared with rSOS 1x. Comparable declines in U-Net-obtained AUC relative to rSOS 1x were also observed at acceleration factor of eight. These results demonstrate that AI-based accelerated MR reconstruction may produce visually pleasing images but may not achieve performance comparable to that of rSOS 1x. The proposed DLMO approach may be employed to characterize the discriminative efficacy of AI-based undersampled reconstruction in MRI.

Figures (5)

• Figure 1: The structure of the proposed DLMO.
• Figure 2: The schematic of the evaluation workflow.
• Figure 3: Examples of objects and reconstructions, including (a) a singlet example and (b) a doublet example.
• Figure 4: AUC values obtained by DLMO with rSOS and U-Net reconstructions at acceleration factors of (a) $4 \times$ and (b) $8 \times$, with 4,000 singlet and 4,000 doublet samples. Note that signal intensity was 0.7 at an acceleration factor of $4 \times$ and 1.3 at $8 \times$.
• Figure 5: Representative reconstructions at an acceleration factor of $8 \times$. (a) rSOS ($1 \times$), rSOS ($8 \times$), and U-Net ($8 \times$) reconstructions without any inserted signals. (b) singlet-signal examples reconstructed under the same conditions. In (b), the U-Net reconstruction hallucinated the singlet signal, producing an appearance resembling a doublet signal.