Unleashing the Potential of Synthetic Images: A Study on Histopathology Image Classification

TL;DR

It is shown that synthetic images can effectively augment existing datasets, ultimately improving the performance of the downstream histopathology image classification task and highlighting the importance of selecting an appropriate generative model type and architecture to enhance performance.

Abstract

Histopathology image classification is crucial for the accurate identification and diagnosis of various diseases but requires large and diverse datasets. Obtaining such datasets, however, is often costly and time-consuming due to the need for expert annotations and ethical constraints. To address this, we examine the suitability of different generative models and image selection approaches to create realistic synthetic histopathology image patches conditioned on class labels. Our findings highlight the importance of selecting an appropriate generative model type and architecture to enhance performance. Our experiments over the PCam dataset show that diffusion models are effective for transfer learning, while GAN-generated samples are better suited for augmentation. Additionally, transformer-based generative models do not require image filtering, in contrast to those derived from Convolutional Neural Networks (CNNs), which benefit from realism score-based selection. Therefore, we show that synthetic images can effectively augment existing datasets, ultimately improving the performance of the downstream histopathology image classification task.

Figures (6)

• Figure 1: General overview of the latent diffusion model. First, the VAE converts the pixel space of the input image $x$ into a more compact spatial representation $z$. Next, we apply the diffusion model, which first employs the Gaussian diffusion process to transform the provided data distribution into a Gaussian distribution $z_T$. The denoising model is then trained to remove the added noise taking into account the class label. Finally, using the VAE decoder, the output of the diffusion model is converted back into pixel space.
• Figure 2: Architecture of the employed denoising models (U-Net based and transformer-based diffusion models). Moreover, this image reflects how the conditioning is performed. The label is combined with the timestep, and after embedding them, they are inserted to the model.
• Figure 3: Selection of generated patches with HistoGAN (GAN) and both DDPM models: U-Net-based (U-Net) and Transformer-based (DiT). All these models were trained with the same dataset. Each dual column together represent the image selection approach used, namely none, realism score, and class-based realism score. In addition, these columns contain an image with the absence (odd) and presence (even) of metastatic tissue, respectively. All the generated synthetic images can be found here: https://github.com/LeireBV/Synthetic_histopathology_dataset
• Figure 4: Image quality analysis. Each of the three graphs display the value of an image quality evaluation metric, comparing them depending on the generative model employed and the image selection approach.
• Figure 5: Analysis on the classification task performance. Each of the three graphs display the classification results (AUC) obtained with each generative model employed, comparing them depending on the training and the image selection approach. The baseline represents the result obtained when training the model with real data and no data augmentation (green line). Traditional augmentation depicts the AUC when training the classification model with only real data and traditional data augmentation techniques (blue line).