Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation

TL;DR

Histopath-C introduces a realistic, on-the-fly corruption benchmark for histopathology vision-language models to evaluate test-time adaptation under domain shifts. It also presents LATTE, a transductive, template-ensembling, low-rank adaptation method that uses multiple text prompts and loss-level aggregation to stabilize predictions. Across ten corruption types and multiple pathology VLMs, LATTE delivers strong robustness and often substantial gains over entropy-based baselines and prior pseudo-labeling approaches, while improving zero-shot performance. This benchmark and method advance practical robustness of medical VLMs in real-world histopathology settings.

Abstract

Medical Vision-language models (VLMs) have shown remarkable performances in various medical imaging domains such as histo\-pathology by leveraging pre-trained, contrastive models that exploit visual and textual information. However, histopathology images may exhibit severe domain shifts, such as staining, contamination, blurring, and noise, which may severely degrade the VLM's downstream performance. In this work, we introduce Histopath-C, a new benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology. Our framework dynamically applies corruptions to any available dataset and evaluates Test-Time Adaptation (TTA) mechanisms on the fly. We then propose LATTE, a transductive, low-rank adaptation strategy that exploits multiple text templates, mitigating the sensitivity of histopathology VLMs to diverse text inputs. Our approach outperforms state-of-the-art TTA methods originally designed for natural images across a breadth of histopathology datasets, demonstrating the effectiveness of our proposed design for robust adaptation in histopathology images. Code and data are available at https://github.com/Mehrdad-Noori/Histopath-C.

Figures (7)

• Figure 1: Illustrative examples of real-world corruption artifacts in histopathology slides, as documented in prior literature. The top row shows representative clean (non-corrupted) images with train–test alignment. The bottom rows depict real instances of staining stain_imagesstain_2stain_3, contamination contamination_image_1contamination_image_2, and blurring artifacts blurring_image_4blurring_image_5blurring_image_6, each introducing domain shifts that can significantly degrade model performance. These examples motivate the need for a test-time adaptation benchmark, such as Histopath-C, that simulates realistic perturbations and enables evaluation of adaptation methods in histopathology.
• Figure 2: Representative examples of the ten corruption types introduced in the Histopath-C benchmark, spanning five categories: Staining, Contamination, Blurring, Noise, and Illumination. These synthetic corruptions are designed to mimic real-world perturbations in histopathology and can be dynamically applied to any dataset for robust evaluation.
• Figure 3: The overall framework of LATTE. All templates are used parallely to compute a loss function. a) We use a transductive pseudolabeling module to align the prediction of each image and its corresponding text caption, and the image-wise and text-wise similarities. b) The losses of the different templates are averaged to build the final adaptation loss used to finetune the model. LoRA and normalization layers are crucial components for adaptation.
• Figure 4: Comparison of Text and Loss averaging over several templates.
• Figure 5: Comparison on the updated parameters.