Evaluating the plausibility of synthetic images for improving automated endoscopic stone recognition

TL;DR

The aim is to create plausible diverse kidney stone images that can be used for pre-training models using ex-vivo data, and it is shown that by mixing natural and synthetic images of CCD images, it is possible to train models capable of performing very well on unseen intra-operative data.

Abstract

Currently, the Morpho-Constitutional Analysis (MCA) is the de facto approach for the etiological diagnosis of kidney stone formation, and it is an important step for establishing personalized treatment to avoid relapses. More recently, research has focused on performing such tasks intra-operatively, an approach known as Endoscopic Stone Recognition (ESR). Both methods rely on features observed in the surface and the section of kidney stones to separate the analyzed samples into several sub-groups. However, given the high intra-observer variability and the complex operating conditions found in ESR, there is a lot of interest in using AI for computer-aided diagnosis. However, current AI models require large datasets to attain a good performance and for generalizing to unseen distributions. This is a major problem as large labeled datasets are very difficult to acquire, and some classes of kidney stones are very rare. Thus, in this paper, we present a method based on diffusion as a way of augmenting pre-existing ex-vivo kidney stone datasets. Our aim is to create plausible diverse kidney stone images that can be used for pre-training models using ex-vivo data. We show that by mixing natural and synthetic images of CCD images, it is possible to train models capable of performing very well on unseen intra-operative data. Our results show that is possible to attain an improvement of 10% in terms of accuracy compared to a baseline model pre-trained only on ImageNet. Moreover, our results show an improvement of 6% for surface images and 10% for section images compared to a model train on CCD images only, which demonstrates the effectiveness of using synthetic images.

Figures (6)

• Figure 1: Examples of synthetic image generation for surface (external) and section (internal) views of kidney stones. The images were generated using the SinDDM model from a dataset of ex-vivo kidney stone images acquired with a charge-coupled device (CCD) camera. The synthetic images present characteristics highly similar in color and texture to the CCD-camera images.
• Figure 2: Evaluating the plausibility of synthetically generated images for automated endoscopic stone recognition pipeline. (a) Generation of synthetic images: The process begins with a set of standard CCD images that are adjusted to dimensions of 2848$\times$4288 pixels through padding. These images are then used to generate low-resolution synthetic versions (264$\times$200 pixels) using SinDDM. Subsequently, these synthetic images undergo an x4 super-resolution process. Finally, the similarity between the input dataset (CCD camera images) and the output dataset (synthetic images) is evaluated using DeepChecks. To carry out the (b) automatic classification of kidney stones in endoscopic images for six different classes, training of models I and II is required. Model I is trained on the synthetic dataset and weights learned from ImageNet (1st TL step). Then, the endoscopic dataset is trained on Model II along with the weights learned in the synthetic distribution (2nd TL step).
• Figure 3: Examples of ex-vivo kidney stone images acquired with (a) a CCD camera and (b) an endoscope. SEC and SUR stand for section and surface views. The first two rows show whole images and in the bottom rows, the sampled image patches
• Figure 4: Distribution plot for each image property showing the difference between the train and the synthetic image (SUR view).
• Figure 5: Heatmap Comparison between the Train Dataset and Synthetic Dataset.