Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lost in Translation: How Language Re-Aligns Vision for Cross-Species Pathology

Ekansh Arora

Abstract

Foundation models are increasingly applied to computational pathology, yet their behavior under cross-cancer and cross-species transfer remains unspecified. This study investigated how fine-tuning CPath-CLIP affects cancer detection under same-cancer, cross-cancer, and cross-species conditions using whole-slide image patches from canine and human histopathology. Performance was measured using area under the receiver operating characteristic curve (AUC). Few-shot fine-tuning improved same-cancer (64.9% to 72.6% AUC) and cross-cancer performance (56.84% to 66.31% AUC). Cross-species evaluation revealed that while tissue matching enables meaningful transfer, performance remains below state-of-the-art benchmarks (H-optimus-0: 84.97% AUC), indicating that standard vision-language alignment is suboptimal for cross-species generalization. Embedding space analysis revealed extremely high cosine similarity (greater than 0.99) between tumor and normal prototypes. Grad-CAM shows prototype-based models remain domain-locked, while language-guided models attend to conserved tumor morphology. To address this, we introduce Semantic Anchoring, which uses language to provide a stable coordinate system for visual features. Ablation studies reveal that benefits stem from the text-alignment mechanism itself, regardless of text encoder complexity. Benchmarking against H-optimus-0 shows that CPath-CLIP's failure stems from intrinsic embedding collapse, which text alignment effectively circumvents. Additional gains were observed in same-cancer (8.52%) and cross-cancer classification (5.67%). We identified a previously uncharacterized failure mode: semantic collapse driven by species-dominated alignment rather than missing visual information. These results demonstrate that language acts as a control mechanism, enabling semantic re-interpretation without retraining.

Lost in Translation: How Language Re-Aligns Vision for Cross-Species Pathology

Abstract

Foundation models are increasingly applied to computational pathology, yet their behavior under cross-cancer and cross-species transfer remains unspecified. This study investigated how fine-tuning CPath-CLIP affects cancer detection under same-cancer, cross-cancer, and cross-species conditions using whole-slide image patches from canine and human histopathology. Performance was measured using area under the receiver operating characteristic curve (AUC). Few-shot fine-tuning improved same-cancer (64.9% to 72.6% AUC) and cross-cancer performance (56.84% to 66.31% AUC). Cross-species evaluation revealed that while tissue matching enables meaningful transfer, performance remains below state-of-the-art benchmarks (H-optimus-0: 84.97% AUC), indicating that standard vision-language alignment is suboptimal for cross-species generalization. Embedding space analysis revealed extremely high cosine similarity (greater than 0.99) between tumor and normal prototypes. Grad-CAM shows prototype-based models remain domain-locked, while language-guided models attend to conserved tumor morphology. To address this, we introduce Semantic Anchoring, which uses language to provide a stable coordinate system for visual features. Ablation studies reveal that benefits stem from the text-alignment mechanism itself, regardless of text encoder complexity. Benchmarking against H-optimus-0 shows that CPath-CLIP's failure stems from intrinsic embedding collapse, which text alignment effectively circumvents. Additional gains were observed in same-cancer (8.52%) and cross-cancer classification (5.67%). We identified a previously uncharacterized failure mode: semantic collapse driven by species-dominated alignment rather than missing visual information. These results demonstrate that language acts as a control mechanism, enabling semantic re-interpretation without retraining.
Paper Structure (22 sections, 1 equation, 6 figures, 7 tables)

This paper contains 22 sections, 1 equation, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Model Architecture
  3. Control Experiments
  4. Datasets
  5. Preprocessing Pipeline
  6. Experimental Design
  7. Evaluation Metrics
  8. Semantic Anchoring Mechanism
  9. Results
  10. Dataset Summary and Experimental Scope
  11. Overall Performance Across Transfer Settings
  12. Same-Cancer Transfer
  13. Cross-Cancer Transfer
  14. Cross-Species Transfer
  15. Embedding Space Analysis
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Same-Cancer Performance Scaling Across Training Sizes.
  • Figure 2: Cross-Species Comparison of Model Performance
  • Figure 3: Cosine Similarity Heatmap Across Domains
  • Figure 4: Grad-CAM Visualization: Canine Breast Carcinoma
  • Figure 5: Grad-CAM Visualization: Canine Mast Cell Tumor
  • ...and 1 more figures