Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PA-MIL: Phenotype-Aware Multiple Instance Learning Guided by Language Prompting and Genotype-to-Phenotype Relationships

Zekang Yang, Hong Liu, Xiangdong Wang

TL;DR

PA-MIL tackles the challenge of interpretable cancer subtyping from pathology WSIs by constructing a phenotype knowledge base and using language prompting to identify cancer-related phenotypes. It couples this with a genotype-to-phenotype network trained on transcriptomics to provide multi-level guidance, enabling phenotype saliency-based subtyping. Across NSCLC and RCC tasks, PA-MIL achieves competitive performance while delivering phenotype-centered evidence and interpretable genotype-phenotype pathways, validated through cohort- and case-level analyses. The work advances reliable, ante-hoc interpretability in computational pathology with practical implications for clinical decision support.

Abstract

Deep learning has been extensively researched in the analysis of pathology whole-slide images (WSIs). However, most existing methods are limited to providing prediction interpretability by locating the model's salient areas in a post-hoc manner, failing to offer more reliable and accountable explanations. In this work, we propose Phenotype-Aware Multiple Instance Learning (PA-MIL), a novel ante-hoc interpretable framework that identifies cancer-related phenotypes from WSIs and utilizes them for cancer subtyping. To facilitate PA-MIL in learning phenotype-aware features, we 1) construct a phenotype knowledge base containing cancer-related phenotypes and their associated genotypes. 2) utilize the morphological descriptions of phenotypes as language prompting to aggregate phenotype-related features. 3) devise the Genotype-to-Phenotype Neural Network (GP-NN) grounded in genotype-to-phenotype relationships, which provides multi-level guidance for PA-MIL. Experimental results on multiple datasets demonstrate that PA-MIL achieves competitive performance compared to existing MIL methods while offering improved interpretability. PA-MIL leverages phenotype saliency as evidence and, using a linear classifier, achieves competitive results compared to state-of-the-art methods. Additionally, we thoroughly analyze the genotype-phenotype relationships, as well as cohort-level and case-level interpretability, demonstrating the reliability and accountability of PA-MIL.

PA-MIL: Phenotype-Aware Multiple Instance Learning Guided by Language Prompting and Genotype-to-Phenotype Relationships

TL;DR

PA-MIL tackles the challenge of interpretable cancer subtyping from pathology WSIs by constructing a phenotype knowledge base and using language prompting to identify cancer-related phenotypes. It couples this with a genotype-to-phenotype network trained on transcriptomics to provide multi-level guidance, enabling phenotype saliency-based subtyping. Across NSCLC and RCC tasks, PA-MIL achieves competitive performance while delivering phenotype-centered evidence and interpretable genotype-phenotype pathways, validated through cohort- and case-level analyses. The work advances reliable, ante-hoc interpretability in computational pathology with practical implications for clinical decision support.

Abstract

Deep learning has been extensively researched in the analysis of pathology whole-slide images (WSIs). However, most existing methods are limited to providing prediction interpretability by locating the model's salient areas in a post-hoc manner, failing to offer more reliable and accountable explanations. In this work, we propose Phenotype-Aware Multiple Instance Learning (PA-MIL), a novel ante-hoc interpretable framework that identifies cancer-related phenotypes from WSIs and utilizes them for cancer subtyping. To facilitate PA-MIL in learning phenotype-aware features, we 1) construct a phenotype knowledge base containing cancer-related phenotypes and their associated genotypes. 2) utilize the morphological descriptions of phenotypes as language prompting to aggregate phenotype-related features. 3) devise the Genotype-to-Phenotype Neural Network (GP-NN) grounded in genotype-to-phenotype relationships, which provides multi-level guidance for PA-MIL. Experimental results on multiple datasets demonstrate that PA-MIL achieves competitive performance compared to existing MIL methods while offering improved interpretability. PA-MIL leverages phenotype saliency as evidence and, using a linear classifier, achieves competitive results compared to state-of-the-art methods. Additionally, we thoroughly analyze the genotype-phenotype relationships, as well as cohort-level and case-level interpretability, demonstrating the reliability and accountability of PA-MIL.
Paper Structure (21 sections, 7 equations, 7 figures, 5 tables)

This paper contains 21 sections, 7 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multiple Instance Learning
  4. Interpretability Methods
  5. Method
  6. Phenotype Knowledge Base
  7. Phenotype-Aware Multiple Instance Learning
  8. Genotype-to-Phenotype Neural Network
  9. Transcriptomics-Guided Phenotype Learning
  10. Experiment and Reseults
  11. Dataset and Implementation Details
  12. Comparisons with State-of-the-Art
  13. Ablation Studies
  14. Interpretability Analysis
  15. Genotype-to-Phenotype Relationsips
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Comparison of the diagnostic process. (a) The diagnostic process of pathologists. (b) The predictive pathway and diagnostic evidence of PA-MIL.
  • Figure 2: Overview of proposed PA-MIL and training method. Step 1: Constructing phenotypic knowledge base. Step 2a: Training genotype-to-phenotype neural network. Step 2b: Training phenotype-aware multiple instance learning with transcriptomic guidance.
  • Figure 3: Heatmaps of phenotypic saliency scores under different activation functions. (Left: Layer normalization, Right: LeakyReLU) The heatmap illustrates the saliency scores of various phenotypes across 1372 samples from the CPTAC-NSCLC. Samples 1-683 are lung adenocarcinoma, and samples 684-1372 are lung squamous cell carcinoma. A higher value indicates a greater saliency score.
  • Figure 4: Relationships among genotypes, phenotypes, and cancer subtypes. Thicker lines means greater contributions. The top-4 genes contributing to each phenotype are shown.
  • Figure 5: Cohort interpretability analysis.
  • ...and 2 more figures