CosSIF: Cosine similarity-based image filtering to overcome low inter-class variation in synthetic medical image datasets

TL;DR

The experimental results reveal that the utilization of either the FAGT or FBGT method reduces low inter-class variation in clinical image classification datasets and enables GANs to generate synthetic images with greater discriminative features.

Abstract

Crafting effective deep learning models for medical image analysis is a complex task, particularly in cases where the medical image dataset lacks significant inter-class variation. This challenge is further aggravated when employing such datasets to generate synthetic images using generative adversarial networks (GANs), as the output of GANs heavily relies on the input data. In this research, we propose a novel filtering algorithm called Cosine Similarity-based Image Filtering (CosSIF). We leverage CosSIF to develop two distinct filtering methods: Filtering Before GAN Training (FBGT) and Filtering After GAN Training (FAGT). FBGT involves the removal of real images that exhibit similarities to images of other classes before utilizing them as the training dataset for a GAN. On the other hand, FAGT focuses on eliminating synthetic images with less discriminative features compared to real images used for training the GAN. Experimental results reveal that employing either the FAGT or FBGT method with modern transformer and convolutional-based networks leads to substantial performance gains in various evaluation metrics. FAGT implementation on the ISIC-2016 dataset surpasses the baseline method in terms of sensitivity by 1.59% and AUC by 1.88%. Furthermore, for the HAM10000 dataset, applying FABT outperforms the baseline approach in terms of recall by 13.75%, and with the sole implementation of FAGT, achieves a maximum accuracy of 94.44%.

Figures (13)

• Figure 1: The illustration depicts the pipeline of our research, which involves identifying the minority class from the dataset, oversampling through GAN and transformation techniques, the adoption of our proposed FBGT and FAGT methods to mitigate low inter-class variation by leveraging our novel CosSIF algorithm, and ultimately training classifiers using the augmented dataset. In the case of multiple minority classes, the pipeline is repeated until the classifier training stage. It is recommended to view the illustration in color.
• Figure 2: The illustration portrays the step-by-step process of the CosSIF algorithm. It commences by selecting the target class ${T^{[c_2]}}$ and a set $X$ comprising secondary classes. Subsequently, the images within the selected classes undergo resizing to 64x64 pixels. Following this, similarity scores are calculated for each image in ${T^{[c_2]}}$ by comparing them to all images in $X$. For each image in ${T^{[c_2]}}$, a record $\eta$ is created to store individual similarity scores $I$ and their corresponding image identifiers. The record $\eta$ is then sorted in descending order, with the first entry representing the maximum similarity score $I_{max}$. Once the similarity calculation for all images in ${T^{[c_2]}}$ has been completed, the resulting set of records $R$ is obtained. Finally, $R$ is sorted in ascending order based on the maximum similarity score $I_{max}$, thereby concluding the similarity calculation process.
• Figure 3: The illustration in (a) depicts the process of calculating the cosine similarity between two images. It begins by dividing a color image into its three RGB (red, green, blue) layers, where each layer is represented as a square matrix. Each layer contains different pixel values, which are then normalized by dividing each pixel by 255. Next, all layers are flattened into a vector. Considering that each image has a resolution of 64x64 pixels and consists of 3 layers, the resulting vector dimension becomes 1x12288. This procedure is repeated for both Image 1 and Image 2, resulting in two vectors, $\mathbf{u}$ and $\mathbf{v}$, respectively. The cosine similarity between these two vectors is then calculated. The graphs in (b) illustrate that as the similarity between vectors $\mathbf{u}$ and $\mathbf{v}$ increases, the angle $\theta$ between them becomes smaller, and vice versa.
• Figure 4: A visual representation of the correlation between the total number of images, $p$, in the target class $\textbf{T}^{[c]}$, and the total number of images, $f$, in the filtered target class $\textbf{T}^{[c]}_{filtered}$, employing the FAGT method, demonstrates that the extent of necessary filtering is influenced by the GAN's capacity to generate synthetic images that closely resemble the real images used to train the GAN.
• Figure 5: The visual depictions of images from various classes indicate low inter-class variation both in the ISIC-2016 dataset, as shown in (a), and the HAM10000 dataset, as shown in (b).