Deep Learning for Breast MRI Style Transfer with Limited Training Data

TL;DR

The paper addresses variability in breast MRI styles arising from different scanners and protocols. It introduces StyleMapper, a disentangled-content/style model trained with simulated random style transformations to enable arbitrary unseen style transfer using limited data. StyleMapper unifies encoders and a decoder, uses a most representative target style code for one-shot transfer, and employs a cross-domain reconstruction triplet loss to enhance generalization across styles unseen during training. The method is demonstrated on breast MRI with both simulated and real scanner styles (GE to Siemens), achieving effective style alignment and enabling downstream tasks such as classification and detection; results indicate practical potential for cross-scanner standardization in medical imaging.

Abstract

In this work we introduce a novel medical image style transfer method, StyleMapper, that can transfer medical scans to an unseen style with access to limited training data. This is made possible by training our model on unlimited possibilities of simulated random medical imaging styles on the training set, making our work more computationally efficient when compared with other style transfer methods. Moreover, our method enables arbitrary style transfer: transferring images to styles unseen in training. This is useful for medical imaging, where images are acquired using different protocols and different scanner models, resulting in a variety of styles that data may need to be transferred between. Methods: Our model disentangles image content from style and can modify an image's style by simply replacing the style encoding with one extracted from a single image of the target style, with no additional optimization required. This also allows the model to distinguish between different styles of images, including among those that were unseen in training. We propose a formal description of the proposed model. Results: Experimental results on breast magnetic resonance images indicate the effectiveness of our method for style transfer. Conclusion: Our style transfer method allows for the alignment of medical images taken with different scanners into a single unified style dataset, allowing for the training of other downstream tasks on such a dataset for tasks such as classification, object detection and others.

Figures (7)

• Figure 1: A comparison of the effects of the seven different image transformation functions that we use (Section \ref{['sec:transforms']} on a DCE-MRI breast scan, with randomized transfer function parameters fixed to the means of their sampling distributions.
• Figure 2: StyleMapper: Our novel architecture used for style transfer. Solid arrows indicate encoding operations, and dashed lines indicate pairs of codes that should be optimally equivalent (Equation \ref{['eq:modelBsamelatentloss']}), with the model trained to achieve as such. The decoder/generator $G$ is not pictured, as it receives input of various combinations of all of the pictured style and content codes.
• Figure 3: One-shot style transfer with various target styles: Qualitative Results. See Section \ref{['sec:exp:oneshot']}. Transferring a set of 25 MR test images $\{X_{\mathrm{test}}\}$ (top row) to different target styles not seen in training $\{X^{s:T}_{\mathrm{test}}\}$ (bottom row), with target style code obtained from a single test image of the target style. The transferred images are compared to the "ground truth" $\{T(X_{\mathrm{test}})\}$ (middle row) of the images directly transformed by the target style's corresponding transformation function $T$. From left to right, the target styles/transformations are the fixed log, gamma/power-law and exp transformations, as described in Section \ref{['sec:exp:oneshot']}, and for each style, three random images are visualized. Accompanying quantitative results in Figure \ref{['fig:fewshot_quant']}.
• Figure 4: One-shot style transfer with various target styles: Quantitative Results. See Section \ref{['sec:exp:oneshot']}. Mean absolute error (MAE) between style transferred images $\{X^{s:T}_{\mathrm{test}}\}$ and "ground truth" transformed images $\{T(X_{\mathrm{test}})\}$, indicating performance of style transfer, with respect to number of target style images $N_\mathrm{target}$ used to compute the most representative target style code that is used to perform style transfer. Accompanying qualitative results in Figure \ref{['fig:fewshot_qual']}.
• Figure 5: MRI Style Transfer to Unseen Scanner Style. Results (right column) of transferring GE scanner MR Images (center column) to the Siemens scanner style unseen in training (left column).