Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Anatomical Conditioning for Contrastive Unpaired Image-to-Image Translation of Optical Coherence Tomography Images

Marc S. Seibel, Hristina Uzunova, Timo Kepp, Heinz Handels

TL;DR

This work tackles cross-device OCT image harmonization under unpaired data by extending contrastive unpaired I2I translation with anatomical conditioning. The proposed ACCUT framework adds a segmentation decoder that guides the style transfer, sharing an encoder and fusing multi-resolution segmentation features with the style Decoder, and introduces four operational modes via loss-weighting. Empirical results on Spectralis-OCT and Home-OCT data show that ACCUT_s (and ACCUT_s,t) improves downstream segmentation performance and image similarity compared to CycleGAN and CUT, while ACCUT_t alone is less effective. The approach reduces semantic hallucination and holds promise for disease monitoring across OCT devices, with potential applicability to AMD biomarkers like SRF and PED in unsupervised domain adaptation settings.

Abstract

For a unified analysis of medical images from different modalities, data harmonization using image-to-image (I2I) translation is desired. We study this problem employing an optical coherence tomography (OCT) data set of Spectralis-OCT and Home-OCT images. I2I translation is challenging because the images are unpaired, and a bijective mapping does not exist due to the information discrepancy between both domains. This problem has been addressed by the Contrastive Learning for Unpaired I2I Translation (CUT) approach, but it reduces semantic consistency. To restore the semantic consistency, we support the style decoder using an additional segmentation decoder. Our approach increases the similarity between the style-translated images and the target distribution. Importantly, we improve the segmentation of biomarkers in Home-OCT images in an unsupervised domain adaptation scenario. Our data harmonization approach provides potential for the monitoring of diseases, e.g., age related macular disease, using different OCT devices.

Anatomical Conditioning for Contrastive Unpaired Image-to-Image Translation of Optical Coherence Tomography Images

TL;DR

This work tackles cross-device OCT image harmonization under unpaired data by extending contrastive unpaired I2I translation with anatomical conditioning. The proposed ACCUT framework adds a segmentation decoder that guides the style transfer, sharing an encoder and fusing multi-resolution segmentation features with the style Decoder, and introduces four operational modes via loss-weighting. Empirical results on Spectralis-OCT and Home-OCT data show that ACCUT_s (and ACCUT_s,t) improves downstream segmentation performance and image similarity compared to CycleGAN and CUT, while ACCUT_t alone is less effective. The approach reduces semantic hallucination and holds promise for disease monitoring across OCT devices, with potential applicability to AMD biomarkers like SRF and PED in unsupervised domain adaptation settings.

Abstract

For a unified analysis of medical images from different modalities, data harmonization using image-to-image (I2I) translation is desired. We study this problem employing an optical coherence tomography (OCT) data set of Spectralis-OCT and Home-OCT images. I2I translation is challenging because the images are unpaired, and a bijective mapping does not exist due to the information discrepancy between both domains. This problem has been addressed by the Contrastive Learning for Unpaired I2I Translation (CUT) approach, but it reduces semantic consistency. To restore the semantic consistency, we support the style decoder using an additional segmentation decoder. Our approach increases the similarity between the style-translated images and the target distribution. Importantly, we improve the segmentation of biomarkers in Home-OCT images in an unsupervised domain adaptation scenario. Our data harmonization approach provides potential for the monitoring of diseases, e.g., age related macular disease, using different OCT devices.
Paper Structure (7 sections, 2 equations, 4 figures, 1 table)

This paper contains 7 sections, 2 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Experiments
  4. ACCUT for Domain Adaptation
  5. Ablation Study
  6. Image Similarity
  7. Conclusion

Figures (4)

  • Figure 1: Anatomical conditioning for style transfer. The segmentation decoder $f_M$ provides shape information to the style decoder $f_S$. The red lines denote that optimizing the style transfer loss does not update the segmentation decoder (no backpropagation).
  • Figure 2: Style transfer and simultaneous segmentation with ACCUT. From left to right, we show a Home-OCT image from the target domain, a Spectralis image from the source domain with its ground truth segmentation, the Spectralis image translated to the target domain and its corresponding segmentation using ACCUT$_{s}$ and ACCUT$_{t}$, respectively. Training the segmentation decoder with only target domain supervision results in bad style transfer for the PED (red circle).
  • Figure 3: Ablation study. A non-matching input is used for the mask decoder, and its features are concatenated to the style decoder. The translated images contain the images from both input images, which shows that the style decoder uses the information from the mask decoder.
  • Figure 4: Qualitative comparison of the image translation methods.