Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conditional Random Fields for Interactive Refinement of Histopathological Predictions

Tiffanie Godelaine, Maxime Zanella, Karim El Khoury, Saïd Mahmoudi, Benoît Macq, Christophe De Vleeschouwer

TL;DR

The paper tackles refining zero-shot patch predictions from histology-focused foundation models by adapting Conditional Random Fields to histology with no additional training. It introduces HistoCRF, which uses unary potentials derived from VLM embeddings and two novel pairwise terms—diversity and annotation-guided—to propagate and align labels across patches under a mean-field inference framework. The approach supports a practical HITL workflow, delivering real-time updates and substantial accuracy gains across five datasets, including up to 5.1% improvement with 100 annotations and 32.6% in HITL scenarios. This work provides a scalable, annotation-friendly refinement mechanism that extends CRFs to histology, enabling more accurate and interactive interpretation of whole-slide image patches.

Abstract

Assisting pathologists in the analysis of histopathological images has high clinical value, as it supports cancer detection and staging. In this context, histology foundation models have recently emerged. Among them, Vision-Language Models (VLMs) provide strong yet imperfect zero-shot predictions. We propose to refine these predictions by adapting Conditional Random Fields (CRFs) to histopathological applications, requiring no additional model training. We present HistoCRF, a CRF-based framework, with a novel definition of the pairwise potential that promotes label diversity and leverages expert annotations. We consider three experiments: without annotations, with expert annotations, and with iterative human-in-the-loop annotations that progressively correct misclassified patches. Experiments on five patch-level classification datasets covering different organs and diseases demonstrate average accuracy gains of 16.0% without annotations and 27.5% with only 100 annotations, compared to zero-shot predictions. Moreover, integrating a human in the loop reaches a further gain of 32.6% with the same number of annotations. The code will be made available on https://github.com/tgodelaine/HistoCRF.

Conditional Random Fields for Interactive Refinement of Histopathological Predictions

TL;DR

The paper tackles refining zero-shot patch predictions from histology-focused foundation models by adapting Conditional Random Fields to histology with no additional training. It introduces HistoCRF, which uses unary potentials derived from VLM embeddings and two novel pairwise terms—diversity and annotation-guided—to propagate and align labels across patches under a mean-field inference framework. The approach supports a practical HITL workflow, delivering real-time updates and substantial accuracy gains across five datasets, including up to 5.1% improvement with 100 annotations and 32.6% in HITL scenarios. This work provides a scalable, annotation-friendly refinement mechanism that extends CRFs to histology, enabling more accurate and interactive interpretation of whole-slide image patches.

Abstract

Assisting pathologists in the analysis of histopathological images has high clinical value, as it supports cancer detection and staging. In this context, histology foundation models have recently emerged. Among them, Vision-Language Models (VLMs) provide strong yet imperfect zero-shot predictions. We propose to refine these predictions by adapting Conditional Random Fields (CRFs) to histopathological applications, requiring no additional model training. We present HistoCRF, a CRF-based framework, with a novel definition of the pairwise potential that promotes label diversity and leverages expert annotations. We consider three experiments: without annotations, with expert annotations, and with iterative human-in-the-loop annotations that progressively correct misclassified patches. Experiments on five patch-level classification datasets covering different organs and diseases demonstrate average accuracy gains of 16.0% without annotations and 27.5% with only 100 annotations, compared to zero-shot predictions. Moreover, integrating a human in the loop reaches a further gain of 32.6% with the same number of annotations. The code will be made available on https://github.com/tgodelaine/HistoCRF.
Paper Structure (11 sections, 8 equations, 1 figure, 4 tables, 1 algorithm)

This paper contains 11 sections, 8 equations, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Method
  4. Unary potentials
  5. Pairwise potentials
  6. Experiments
  7. Experiment 1: Without annotations (|A|=1)
  8. Experiment 2: With expert annotations (|A|≥1)
  9. Experiment 3: Human-in-the-loop
  10. Ablation studies: Impact of pairwise potential terms
  11. Conclusion

Figures (1)

  • Figure 1: Refinement of histopathological zero-shot predictions on three WSIs of breast cancer tissue datasetbach using the proposed method applied to patches extracted from each WSI in the HITL setting. After each prediction, the pathologist annotates a region indicated by the white arrow. These annotations then guide the refinement step leading to improved predictions.