Improved HER2 Tumor Segmentation with Subtype Balancing using Deep Generative Networks

TL;DR

The suitability of Generative Adversarial Networks (GANs) and especially diffusion models to create realistic images based on subtype-conditioning for the use case of HER2-stained histopathology is shown and the capability of diffusion Models to conditionally inpaint HER2 tumor areas with modified subtypes is shown.

Abstract

Tumor segmentation in histopathology images is often complicated by its composition of different histological subtypes and class imbalance. Oversampling subtypes with low prevalence features is not a satisfactory solution since it eventually leads to overfitting. We propose to create synthetic images with semantically-conditioned deep generative networks and to combine subtype-balanced synthetic images with the original dataset to achieve better segmentation performance. We show the suitability of Generative Adversarial Networks (GANs) and especially diffusion models to create realistic images based on subtype-conditioning for the use case of HER2-stained histopathology. Additionally, we show the capability of diffusion models to conditionally inpaint HER2 tumor areas with modified subtypes. Combining the original dataset with the same amount of diffusion-generated images increased the tumor Dice score from 0.833 to 0.854 and almost halved the variance between the HER2 subtype recalls. These results create the basis for more reliable automatic HER2 analysis with lower performance variance between individual HER2 subtypes.

Figures (6)

• Figure 1: The her2 tumor subtypes present in the her2 annotations, including two dcis examples with tissue corresponding to different her2 subtypes.
• Figure 2: Distribution of the different her2 subtypes across the annotated tumor regions for the combined training and validation set, as well as for the test set.
• Figure 3: Illustration of how the synthetic datasets are created. An image and a corresponding label mask are sampled from the real dataset, and the subtype label of each tumor tissue instance is randomly modified. With the new label masks, synthetic images are created using a gan, a diffusion model or diffusion inpainting. The generated images, together with their new label masks, are added to the synthetic dataset.
• Figure 4: Visual comparison of images created by generative networks.
• Figure 5: Boxplots of the tumor Dice score and the subtype variance for different configurations. Mean and standard deviation are visualized with the boxplot, while the whiskers mark the minimum and maximum values.