Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Histopathology Slide Indexing and Search: Are We There Yet?

Helen H. Shang, Mohammad Sadegh Nasr, Jai Prakash Veerla, Parisa Boodaghi Malidarreh, MD Jillur Rahman Saurav, Amir Hajighasemi, Manfred Huber, Chace Moleta, Jitin Makker, Jacob M. Luber

TL;DR

The study confronts the question of whether current histopathology slide search engines are ready for clinical use by externally validating four approaches on unseen cases and diverse datasets. It employs mosaic-based indexing, barcoding, VQ-VAE indexing, contrastive learning, and hypergraph retrieval to compare Yottixel, SISH, RetCCL, and HSHR under a unified evaluation framework. The findings show no method delivers consistently reliable performance across site and tissue types, though RetCCL often performs best on tissue and subtype tasks, with notable generalizability gaps and substantial expert variability in patch-level assessments. The paper concludes with a practical set of minimal requirements for clinical deployment, emphasizing richer representations, systematic evaluation, robustness to unseen data, replicability, and computational efficiency to support safer and more effective adoption in pathology settings.

Abstract

The search and retrieval of digital histopathology slides is an important task that has yet to be solved. In this case study, we investigate the clinical readiness of three state-of-the-art histopathology slide search engines, Yottixel, SISH, and RetCCL, on three patients with solid tumors. We provide a qualitative assessment of each model's performance in providing retrieval results that are reliable and useful to pathologists. We found that all three image search engines fail to produce consistently reliable results and have difficulties in capturing granular and subtle features of malignancy, limiting their diagnostic accuracy. Based on our findings, we also propose a minimal set of requirements to further advance the development of accurate and reliable histopathology image search engines for successful clinical adoption.

Histopathology Slide Indexing and Search: Are We There Yet?

TL;DR

The study confronts the question of whether current histopathology slide search engines are ready for clinical use by externally validating four approaches on unseen cases and diverse datasets. It employs mosaic-based indexing, barcoding, VQ-VAE indexing, contrastive learning, and hypergraph retrieval to compare Yottixel, SISH, RetCCL, and HSHR under a unified evaluation framework. The findings show no method delivers consistently reliable performance across site and tissue types, though RetCCL often performs best on tissue and subtype tasks, with notable generalizability gaps and substantial expert variability in patch-level assessments. The paper concludes with a practical set of minimal requirements for clinical deployment, emphasizing richer representations, systematic evaluation, robustness to unseen data, replicability, and computational efficiency to support safer and more effective adoption in pathology settings.

Abstract

The search and retrieval of digital histopathology slides is an important task that has yet to be solved. In this case study, we investigate the clinical readiness of three state-of-the-art histopathology slide search engines, Yottixel, SISH, and RetCCL, on three patients with solid tumors. We provide a qualitative assessment of each model's performance in providing retrieval results that are reliable and useful to pathologists. We found that all three image search engines fail to produce consistently reliable results and have difficulties in capturing granular and subtle features of malignancy, limiting their diagnostic accuracy. Based on our findings, we also propose a minimal set of requirements to further advance the development of accurate and reliable histopathology image search engines for successful clinical adoption.
Paper Structure (14 sections, 3 figures, 3 tables, 6 algorithms)

This paper contains 14 sections, 3 figures, 3 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Methods
  3. Search Engines
  4. Database Slides
  5. Test Datasets
  6. Experiments
  7. Results
  8. Tissue and subtype retrieval
  9. Visual review of query results
  10. Reader study
  11. HER2+ prediction
  12. Ablation study
  13. Microscope study
  14. Discussion

Figures (3)

  • Figure 1: A summary of feature extraction and database creation processes proposed by (a) Yottixel kalra_yottixel_2020, (b) SISH chen_fast_2022, (c) RetCCL wang_retccl_2023, and (d) HSHR li_high-order_2023. The feature extractor of Yottixel is switched with KimiaNet riasatian_fine-tuning_2021.
  • Figure 2: Results of site retrieval (left) and sub-type retrieval (right) at slide level for all three test slides. Correct labels are printed in green under query slides. Green border means correct label; red border means wrong label. For details about distances and similarities, see \ref{['subsec:searchengines']}.
  • Figure 3: Results of patch retrieval for two patches from Slide 1. Correct labels are printed in green to the left of query patches. Green border means correct label; red border means wrong label. For details about distances and similarities, see \ref{['subsec:searchengines']}.