Single-subject Multi-contrast MRI Super-resolution via Implicit Neural Representations

TL;DR

The paper tackles the lack of isotropic high-resolution MR data by enabling subject-specific, multi-contrast super-resolution from two LR scans. It introduces a joint implicit neural representation with a split-head architecture that shares anatomy while preserving contrast-specific detail. Mutual Information ($MI$) is used both as an evaluation metric and as an early stopping criterion, with results showing convergence toward an optimum $MI$. Experiments on BraTS, MSSEG, and cMS datasets demonstrate improved PSNR/SSIM/LPIPS and preservation of lesions, indicating potential to reduce scan time and bias in clinical practice.

Abstract

Clinical routine and retrospective cohorts commonly include multi-parametric Magnetic Resonance Imaging; however, they are mostly acquired in different anisotropic 2D views due to signal-to-noise-ratio and scan-time constraints. Thus acquired views suffer from poor out-of-plane resolution and affect downstream volumetric image analysis that typically requires isotropic 3D scans. Combining different views of multi-contrast scans into high-resolution isotropic 3D scans is challenging due to the lack of a large training cohort, which calls for a subject-specific framework. This work proposes a novel solution to this problem leveraging Implicit Neural Representations (INR). Our proposed INR jointly learns two different contrasts of complementary views in a continuous spatial function and benefits from exchanging anatomical information between them. Trained within minutes on a single commodity GPU, our model provides realistic super-resolution across different pairs of contrasts in our experiments with three datasets. Using Mutual Information (MI) as a metric, we find that our model converges to an optimum MI amongst sequences, achieving anatomically faithful reconstruction. Code is available at: https://github.com/jqmcginnis/multi_contrast_inr/

Figures (4)

• Figure 1: Overview of our proposed approach (best viewed in full screen). a) Given a realistic clinical scenario, two MRI contrasts are acquired in complementary 2D views. b) Our proposed INR models both contrast from the supervision available in the 2D scans and, by doing so, learn to transfer knowledge from in-plane measurements to out-of-plane of the other contrast. Although our model is trained on MSELoss only for the observed coordinates, it constructs a continuous function space, converging to an optimum state of mutual information between the contrasts on the global space of $\Omega$. c) Once learned, we can sample an isotropic grid and obtain the anatomically faithful and pathology-preserving super-resolution.
• Figure 2: Qualitative results for MCSR for cMS. The predictions of the split-head INR demonstrate the transfer of anatomical and lesion knowledge from complementing views and sequences. Yellow boxes highlight details recovered by the split-head INR in the out-of-plane reconstructions, where others struggle.
• Figure 3: Convergence of predicted $MI(\hat{I}_1, \hat{I}_2)$ shown in a dashed line to the ground truth state $MI(I_1, I_2)$ shown in solid line for five randomly selected subjects (shown in a different color) for two datasets. Note that initially, the MI between two predicted contrasts is high because of randomly initialized shared weights, and over the training period reaches a plateau close to the true equilibrium.
• Figure 4: (Best viewed in fullscreen.) Qualitative comparisons of different models for a typical subject of the MSSEG (upper part) and BraTS (lower part) dataset. Starting from limited out-of-plane information of the input LR scans, the split-head INR is capable of retrieving recoverable anatomical structures providing truthfulness to its prediction. Exploiting the consistency and mutual anatomical information, the split-head INR can resolve ambiguities in joint reconstruction, as highlighted in yellow boxes, which is impossible if trained in a single contrast setting.