Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Combining Graph Neural Network and Mamba to Capture Local and Global Tissue Spatial Relationships in Whole Slide Images

Ruiwen Ding, Kha-Dinh Luong, Erika Rodriguez, Ana Cristina Araujo Lemos da Silva, William Hsu

TL;DR

This work introduces a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships among the tiles in WSIs and demonstrates its effectiveness in predicting progression-free survival among patients with early-stage lung adenocarcinomas.

Abstract

In computational pathology, extracting spatial features from gigapixel whole slide images (WSIs) is a fundamental task, but due to their large size, WSIs are typically segmented into smaller tiles. A critical aspect of this analysis is aggregating information from these tiles to make predictions at the WSI level. We introduce a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships among the tiles in WSIs. The model's effectiveness was demonstrated in predicting progression-free survival among patients with early-stage lung adenocarcinomas (LUAD). We compared the model with other state-of-the-art methods for tile-level information aggregation in WSIs, including tile-level information summary statistics-based aggregation, multiple instance learning (MIL)-based aggregation, GNN-based aggregation, and GNN-transformer-based aggregation. Additional experiments showed the impact of different types of node features and different tile sampling strategies on the model performance. This work can be easily extended to any WSI-based analysis. Code: https://github.com/rina-ding/gat-mamba.

Combining Graph Neural Network and Mamba to Capture Local and Global Tissue Spatial Relationships in Whole Slide Images

TL;DR

This work introduces a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships among the tiles in WSIs and demonstrates its effectiveness in predicting progression-free survival among patients with early-stage lung adenocarcinomas.

Abstract

In computational pathology, extracting spatial features from gigapixel whole slide images (WSIs) is a fundamental task, but due to their large size, WSIs are typically segmented into smaller tiles. A critical aspect of this analysis is aggregating information from these tiles to make predictions at the WSI level. We introduce a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships among the tiles in WSIs. The model's effectiveness was demonstrated in predicting progression-free survival among patients with early-stage lung adenocarcinomas (LUAD). We compared the model with other state-of-the-art methods for tile-level information aggregation in WSIs, including tile-level information summary statistics-based aggregation, multiple instance learning (MIL)-based aggregation, GNN-based aggregation, and GNN-transformer-based aggregation. Additional experiments showed the impact of different types of node features and different tile sampling strategies on the model performance. This work can be easily extended to any WSI-based analysis. Code: https://github.com/rina-ding/gat-mamba.
Paper Structure (22 sections, 3 equations, 5 figures, 4 tables, 2 algorithms)

This paper contains 22 sections, 3 equations, 5 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Survival analysis in computational pathology
  4. Graph-based approaches in computational pathology
  5. Long sequence modeling
  6. Methods
  7. Graph construction
  8. Model architecture
  9. The GAT branch
  10. The Mamba branch
  11. The GAT-Mamba pipeline
  12. Experiments and Results
  13. Dataset
  14. Implementation details
  15. Evaluation metrics
  16. ...and 7 more sections

Figures (5)

  • Figure 1: The proposed GAT-Mamba pipeline, which consists of WSI tiling (a), node and edge feature extraction (b), graph construction (c), initializing the graph with the extracted node and edge features, and modeling on the graphs. BN: batch normalization. MLP: multi-layer perceptron. +: element-wise summation. N = 16 for positional encodings.
  • Figure 2: Box plots of the 5-fold cross-validation test set C-indices (a) and dynamic AUCs (b) for GAT-Mamba and all baseline models.
  • Figure 3: Results of tile sampling experiments. (a) represents the line graphs visualizing the average C-index and its standard deviation across different percentages of tiles sampled or when using only aggressive or when using only less aggressive tiles, using UNI node features. (b) represents a bar graph showing the macro-average of the average C-index across 5, 10, 20, 30, 60, and 100 percent sampling for all six types of node features, and (c) represents the macro-range version.
  • Figure 4: Visualization of characteristics of GAT-Mamba predicted risk groups. (a) represents the Kaplan–Meier curves of low and high-risk groups where the log-rank test p-value indicates a statistically significant difference in the progression-free survival distribution of the two groups. (b) represents the overall distribution of predominant histologic subtypes in low (left) and high (right) risk patients. (c) represents the distribution of the percentage of solid tiles in each patient in low and high-risk groups. (d)(e) show the distribution of four hand-crafted features between low and high-risk groups, with (d) representing the number of lymphocytes divided by the number of all nuclei, (e) representing the TIL abundance score shaban2019novel.
  • Figure 5: Characteristics of tiles for model-predicted false negative (FN), true positive (TP), and false positive (FP) patients. (a) shows the distribution of the percentage of non-tumor tiles. (b) shows the distribution of the percentage of solid tiles. (c) shows the distribution of the total number of tiles. **** means p $<$ 0.001, ns means p $\geq$ 0.05 using Wilcoxon rank sum test.