Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

S-CycleGAN: Semantic Segmentation Enhanced CT-Ultrasound Image-to-Image Translation for Robotic Ultrasonography

Yuhan Song, Nak Young Chong

TL;DR

The paper addresses data scarcity and anatomical fidelity in ultrasound analysis by introducing S-CycleGAN, a semantic-discriminator–augmented CycleGAN that translates CT slices into ultrasound-like images while preserving key anatomical structures. It expands CycleGAN with segmentation networks as semantic discriminators and uses dual inputs (image and semantic maps) to guide translation, optimizing adversarial, cycle-consistency, and segmentation losses, $$L_{adv}, L_{cycle}, L_{seg}$$. Evaluations on Abdomen-1K CT and Kaggle US simulation datasets demonstrate improved semantic preservation and ultrasound-like texture in synthetic images, suggesting utility for training robotic ultrasound systems. The work highlights practical potential for data augmentation in medical image analysis, while acknowledging the need for quantitative metrics and clinical validation to confirm diagnostic utility.

Abstract

Ultrasound imaging is pivotal in various medical diagnoses due to its non-invasive nature and safety. In clinical practice, the accuracy and precision of ultrasound image analysis are critical. Recent advancements in deep learning are showing great capacity of processing medical images. However, the data hungry nature of deep learning and the shortage of high-quality ultrasound image training data suppress the development of deep learning based ultrasound analysis methods. To address these challenges, we introduce an advanced deep learning model, dubbed S-CycleGAN, which generates high-quality synthetic ultrasound images from computed tomography (CT) data. This model incorporates semantic discriminators within a CycleGAN framework to ensure that critical anatomical details are preserved during the style transfer process. The synthetic images are utilized to enhance various aspects of our development of the robot-assisted ultrasound scanning system. The data and code will be available at https://github.com/yhsong98/ct-us-i2i-translation.

S-CycleGAN: Semantic Segmentation Enhanced CT-Ultrasound Image-to-Image Translation for Robotic Ultrasonography

TL;DR

The paper addresses data scarcity and anatomical fidelity in ultrasound analysis by introducing S-CycleGAN, a semantic-discriminator–augmented CycleGAN that translates CT slices into ultrasound-like images while preserving key anatomical structures. It expands CycleGAN with segmentation networks as semantic discriminators and uses dual inputs (image and semantic maps) to guide translation, optimizing adversarial, cycle-consistency, and segmentation losses, . Evaluations on Abdomen-1K CT and Kaggle US simulation datasets demonstrate improved semantic preservation and ultrasound-like texture in synthetic images, suggesting utility for training robotic ultrasound systems. The work highlights practical potential for data augmentation in medical image analysis, while acknowledging the need for quantitative metrics and clinical validation to confirm diagnostic utility.

Abstract

Ultrasound imaging is pivotal in various medical diagnoses due to its non-invasive nature and safety. In clinical practice, the accuracy and precision of ultrasound image analysis are critical. Recent advancements in deep learning are showing great capacity of processing medical images. However, the data hungry nature of deep learning and the shortage of high-quality ultrasound image training data suppress the development of deep learning based ultrasound analysis methods. To address these challenges, we introduce an advanced deep learning model, dubbed S-CycleGAN, which generates high-quality synthetic ultrasound images from computed tomography (CT) data. This model incorporates semantic discriminators within a CycleGAN framework to ensure that critical anatomical details are preserved during the style transfer process. The synthetic images are utilized to enhance various aspects of our development of the robot-assisted ultrasound scanning system. The data and code will be available at https://github.com/yhsong98/ct-us-i2i-translation.
Paper Structure (18 sections, 4 equations, 6 figures, 1 table, 1 algorithm)

This paper contains 18 sections, 4 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Image-to-image translation
  4. Ultrasound image synthesis
  5. METHODOLOGY
  6. CycleGAN
  7. Proposed semantic segmentation enhanced S-CycleGAN
  8. Generators $G_{CT \to US}$ and $G_{US \to CT}$
  9. Discriminators $D_{US}$ and $D_{CT}$
  10. Segmentation Networks ($S_{US}$ and $S_{CT}$)
  11. Adversarial Loss
  12. Cycle Consistency Loss
  13. Segmentation Loss
  14. EXPERIMENTS and RESULTS
  15. Dataset
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Our robot-assisted ultrasound imaging system
  • Figure 2: CycleGAN for CT-ultrasound translation
  • Figure 3: CT-ultrasound translation using pure CycleGAN
  • Figure 4: Proposed Pipeline
  • Figure 5: CT-to-ultrasound translation example.1
  • ...and 1 more figures