A filtering scheme for confocal laser endomicroscopy (CLE)-video sequences for self-supervised learning

TL;DR

This work tackles the scarcity and redundancy of CLE data for deep learning by introducing CLE-ViFi, a SSIM-based video filtering scheme that de-duplicates CLE video frames. The authors combine self-supervised pretraining on a large unlabeled CLE corpus with a vision transformer backbone (ViT-small) using a DINO-like loss, and evaluate transfer to two CLE-based downstream tasks. They demonstrate that SSL pretraining on CLE data, especially when paired with CLE-ViFi filtering (HAN-ViFi), yields higher test accuracy than ImageNet-based baselines and substantially reduces training time. These findings support the viability of SSL for CLE and offer a practical approach to accelerate model training for intraoperative imaging applications.

Abstract

Confocal laser endomicroscopy (CLE) is a non-invasive, real-time imaging modality that can be used for in-situ, in-vivo imaging and the microstructural analysis of mucous structures. The diagnosis using CLE is, however, complicated by images being hard to interpret for non-experienced physicians. Utilizing machine learning as an augmentative tool would hence be beneficial, but is complicated by the shortage of histopathology-correlated CLE imaging sequences with respect to the plurality of patterns in this domain, leading to overfitting of machine learning models. To overcome this, self-supervised learning (SSL) can be employed on larger unlabeled datasets. CLE is a video-based modality with high inter-frame correlation, leading to a non-stratified data distribution for SSL training. In this work, we propose a filter functionality on CLE video sequences to reduce the dataset redundancy in SSL training and improve SSL training convergence and training efficiency. We use four state-of-the-art baseline networks and a SSL teacher-student network with a vision transformer small backbone for the evaluation. These networks were evaluated on downstream tasks for a sinonasal tumor dataset and a squamous cell carcinoma of the skin dataset. On both datasets, we found the highest test accuracy on the filtered SSL-pretrained model, with 67.48% and 73.52%, both considerably outperforming their non-SSL baselines. Our results show that SSL is an effective method for CLE pretraining. Further, we show that our proposed CLE video filter can be utilized to improve training efficiency in self-supervised scenarios, resulting in a reduction of 67% in training time.

Figures (2)

• Figure 1.1: Overview of the approach: We used self-supervised pre-training on a dataset utilizing our proposed CLE video filter (ViFi) and investigated two downstream tasks (SNT-DS, SCCS-DS).
• Figure 1.2: SSIM feature histogram (left) and ROC-AUC curve for SSIM-based thresholding (right).