IntraStyler: Exemplar-based Style Synthesis for Cross-modality Domain Adaptation

TL;DR

This work addresses cross-modality domain adaptation in medical imaging by tackling intra-domain variability through exemplar-based style synthesis. It introduces IntraStyler, which extends Contrastive Unpaired Translation with a contrastively learned style encoder and dynamic instance normalization to produce exemplar-consistent, diverse target-domain styles without pre-defined sub-domains, plus SLERP-style interpolation for smooth style transitions. The approach yields a well-structured style embedding, enables effective exemplar-guided synthesis, and improves downstream segmentation robustness on CrossMoDA 2023 relative to baselines that rely on limited style variation. The proposed method offers a practical path to more robust UDA in multi-site, multi-scanner medical imaging scenarios and suggests avenues for extending style disentanglement and 3D disentanglement in future work.

Abstract

Image-level domain alignment is the de facto approach for unsupervised domain adaptation, where unpaired image translation is used to minimize the domain gap. Prior studies mainly focus on the domain shift between the source and target domains, whereas the intra-domain variability remains under-explored. To address the latter, an effective strategy is to diversify the styles of the synthetic target domain data during image translation. However, previous methods typically require intra-domain variations to be pre-specified for style synthesis, which may be impractical. In this paper, we propose an exemplar-based style synthesis method named IntraStyler, which can capture diverse intra-domain styles without any prior knowledge. Specifically, IntraStyler uses an exemplar image to guide the style synthesis such that the output style matches the exemplar style. To extract the style-only features, we introduce a style encoder to learn styles discriminatively based on contrastive learning. We evaluate the proposed method on the largest public dataset for cross-modality domain adaptation, CrossMoDA 2023. Our experiments show the efficacy of our method in controllable style synthesis and the benefits of diverse synthetic data for downstream segmentation. Code is available at https://github.com/han-liu/IntraStyler.

Figures (5)

• Figure 1: Left: an illustration of intra-domain variability in UDA tasks. Many sub-domains (scanner type) exist in the target domain (T2 MRI). Right: a comparison of different image translation strategies to generate diverse output styles.
• Figure 2: Illustration of IntraStyler (left) and our contrastive learning setup (right).
• Figure 3: As the SLERP interpolated style (orange arrow) rotates clockwise, the style of the interpolated image also transitions between two exemplar images from left to right.
• Figure 4: Top left: the style embedding space of all unlabeled target domain images. Clustering is done by K-means. Top right: representative MR images selected from two local regions in the style embedding space. It can be seen that the MR images from the same region have consistent styles while having different anatomies, demonstrating the effectiveness of this well-disentangled style latent space. Bottom: the synthesis results of input ceT1 images (left column) with different T2 exemplar images (top row). The exemplars are selected as the most representative sample (denoted by stars) of each cluster. The synthesized T2 images follow the anatomies of their input ceT1 images while preserving the styles of the exemplar T2 images.
• Figure 5: The segmentation performance on the CrossMoDA 2023 validation leaderboard. The results were obtained by training nnU-Net on the synthetic T2 images generated by different synthesis strategies. For ASSD plots, the number of failure cases (i.e., no segmentation for the target structure) of the top 3 methods is displayed in the red box.