Evaluation of Multi-Scale Multiple Instance Learning to Improve Thyroid Cancer Classification

TL;DR

The paper addresses automatic differentiation of thyroid cancer subtypes on gigapixel WSIs, where direct deep learning is impractical. It extends a patch-based MIL pipeline by evaluating three multi-scale aggregation strategies (MC, MA, MM) that combine feature vectors from three patch resolutions. MM yielded the best performance, achieving about 0.88 mean accuracy with modest variability, while MC and MA did not surpass the single-scale baseline. These findings underscore the importance of high-resolution information for FN vs PC differentiation and motivate further scale-aware MIL work with larger datasets and attention-based patch analysis.

Abstract

Thyroid cancer is currently the fifth most common malignancy diagnosed in women. Since differentiation of cancer sub-types is important for treatment and current, manual methods are time consuming and subjective, automatic computer-aided differentiation of cancer types is crucial. Manual differentiation of thyroid cancer is based on tissue sections, analysed by pathologists using histological features. Due to the enormous size of gigapixel whole slide images, holistic classification using deep learning methods is not feasible. Patch based multiple instance learning approaches, combined with aggregations such as bag-of-words, is a common approach. This work's contribution is to extend a patch based state-of-the-art method by generating and combining feature vectors of three different patch resolutions and analysing three distinct ways of combining them. The results showed improvements in one of the three multi-scale approaches, while the others led to decreased scores. This provides motivation for analysis and discussion of the individual approaches.

Figures (3)

• Figure 1: Multi-scale patch extraction process. Starting points of the highest resolution patch were uniformly sampled from the WSI dimensions. This is followed by checks on if the patches show tissue and if the lower resolution patches (which were spatially larger) fit inside the WSI. If they did not, the starting positions for the current patch was moved by $\frac{1}{4}$ of its size towards the image center. This movement was done for both axis.
• Figure 2: This figure shows the different multi-scale cancer differentiation pipelines . After extracting multi-scale patches, features are extracted from these patches by ResNet18 which are uniquely aggregated in MC and MA. MM handles the features of the different scales independently. In MC and MA, these features are then clustered per patch and these clusters are finally aggregated into one bag-of-words per WSI. MM clusters the features of each scale independently and aggregates them into three different histograms. MM then concatenates these bag-of-words which are finally used to train/evaluate SVMs.
• Figure 3: This figure shows the results of the experiments with the x-Axis denoting the different numbers of clusters and methods and the y-Axis denoting the classification accuracy starting from $0.5$. The plots show the different methods with the same setup ($svm: {linear, RBF, optimized}$, $k = {32, 64, 128, 256}$). The mean baseline accuracy is displayed as horizontal blue lines. The bars correspond to the models reached mean accuracy with the bar color denoting the experiment (MC: yello, MA: green, MM: red). The black lines on top of the bars correspond to the standard deviation of the current bar.