Alternative Learning Paradigms for Image Quality Transfer

TL;DR

Experiments identify that the two novel formulations of the IQT problem can avoid bias associated with supervised meth- ods when tested using out-of-distribution data that differs from the distribution of the data the model was trained on.

Abstract

Image Quality Transfer (IQT) aims to enhance the contrast and resolution of low-quality medical images, e.g. obtained from low-power devices, with rich information learned from higher quality images. In contrast to existing IQT methods which adopt supervised learning frameworks, in this work, we propose two novel formulations of the IQT problem. The first approach uses an unsupervised learning framework, whereas the second is a combination of both supervised and unsupervised learning. The unsupervised learning approach considers a sparse representation (SRep) and dictionary learning model, which we call IQT-SRep, whereas the combination of supervised and unsupervised learning approach is based on deep dictionary learning (DDL), which we call IQT-DDL. The IQT-SRep approach trains two dictionaries using a SRep model using pairs of low- and high-quality volumes. Subsequently, the SRep of a low-quality block, in terms of the low-quality dictionary, can be directly used to recover the corresponding high-quality block using the high-quality dictionary. On the other hand, the IQT-DDL approach explicitly learns a high-resolution dictionary to upscale the input volume, while the entire network, including high dictionary generator, is simultaneously optimised to take full advantage of deep learning methods. The two models are evaluated using a low-field magnetic resonance imaging (MRI) application aiming to recover high-quality images akin to those obtained from high-field scanners. Experiments comparing the proposed approaches against state-of-the-art supervised deep learning IQT method (IQT-DL) identify that the two novel formulations of the IQT problem can avoid bias associated with supervised methods when tested using out-of-distribution data that differs from the distribution of the data the model was trained on. This highlights the potential benefit of these novel paradigms for IQT.

Figures (6)

• Figure 1: A schematic diagram of the proposed IQT approach using sparse representation and dictionary learning (IQT-SRep), where ${\mathbf{D}}_h$: High-quality dictionary, ${\mathbf{D}}_\ell$: Low-quality dictionary, ${\mathbf{Y}}$: Low-quality input volume, ${\mathbf{X}}_0$: Initial high-quality volume, $\lambda, \beta$ Regularisation parameters, $\sqrt[3]{m}$: Patch size, $p$: Number of pixel overlap, $s$: Scale, ${\mathbf{y}}$: A patch from the low-quality image ${\mathbf{Y}}$, $\mu$: Mean intensity of the patch ${\mathbf{y}}$, ${\boldsymbol{\alpha}}$: Sparse representation coefficients, ${\boldsymbol{\alpha}}^*$: Optimised sparse representation coefficients, ${\mathbf{F}}$: Transformation matrix, ${\mathbf{x}}$: High-quality patch, and ${\mathbf{X}}^*$: High-quality volume.
• Figure 2: A schematic diagram of IQT using deep dictionary learning. Random noise generates the high-resolution dictionary ${\mathbf{D}}_H$. Then a per-pixel predictor takes as input a concatenation of an encoded code of ${\mathbf{D}}_H$ and an extracted feature. A final image based on ${\mathbf{D}}_H$ is then constructed using predictor output.
• Figure 3: A schematic diagram of training and testing datasets. The two test in distribution datasets (InD1 and InD2) are synthesised using parameters sampled from the same 2D Gaussian distribution used for the training set. In particular, InD1 is synthesised with parameters using a Mahalanobis distance $< 1$, and InD2 with Mahalanobis distance $> 3$, with the constraint of having the SNR higher in WM than in GM, to keep the tissue contrast compatible with T1w. The out-of-distribution (OOD) data set is simulated using parameters sampled from a distribution estimated from ultra-low field T1w images.
• Figure 4: Results using the HCP data set on coronal direction of the three different data distributions InD1, InD2 and OOD (rows) using IQT-SRep, IQT-DDL and IQT-DL. First column shows interpolated low-field image, second to forth columns show image estimate using IQT-DL, IQT-DDL and IQT-SRep, respectively, and fifth column shows original high-field image.
• Figure 5: Absolute errors for results in Figure \ref{['fig:Estimates']}, between gold-standard high-field image (Column 5 of Figure 4), and Column 1: corresponding low-quality image, Column 2: IQT-DL, Column 3: IQT-DDL, and Column 4: IQT-SRep. Columns 5 and 6 show binary maps of regions (in red label) where the IQT-SRep and IQT-DDL, respectively provide closer estimates to the gold-standard high-field images compared to IQT-DL.