Integrative Graph-Transformer Framework for Histopathology Whole Slide Image Representation and Classification

TL;DR

This paper introduces the integrative graph-transformer framework that simultaneously captures the context-aware relational features and global WSI representations through a novel Graph Transformer Integration (GTI) block.

Abstract

In digital pathology, the multiple instance learning (MIL) strategy is widely used in the weakly supervised histopathology whole slide image (WSI) classification task where giga-pixel WSIs are only labeled at the slide level. However, existing attention-based MIL approaches often overlook contextual information and intrinsic spatial relationships between neighboring tissue tiles, while graph-based MIL frameworks have limited power to recognize the long-range dependencies. In this paper, we introduce the integrative graph-transformer framework that simultaneously captures the context-aware relational features and global WSI representations through a novel Graph Transformer Integration (GTI) block. Specifically, each GTI block consists of a Graph Convolutional Network (GCN) layer modeling neighboring relations at the local instance level and an efficient global attention model capturing comprehensive global information from extensive feature embeddings. Extensive experiments on three publicly available WSI datasets: TCGA-NSCLC, TCGA-RCC and BRIGHT, demonstrate the superiority of our approach over current state-of-the-art MIL methods, achieving an improvement of 1.0% to 2.6% in accuracy and 0.7%-1.6% in AUROC.

Figures (1)

• Figure 1: Overview of the proposed method: The process begins with the graph construction module where a graph representation $G = (\mathbf{H}, \mathbf{A})$ is generated for the subsequent backbone network. Here, the $\mathbf{H}$ is the feature matrix and $\mathbf{A}$ denotes the associated adjacent matrix. Within each $l$-th integration block of the backbone, a global attention layer processes $\mathbf{H}^l$ to produce the feature matrix $\mathbf{T}^{l+1}$, and a GCN layer processes both $\mathbf{H}^l$ and $\mathbf{A}$ to update the graph representation $\mathbf{G}^{l+1}$. Finally, the integrated feature $\mathbf{H}^{l+1}$ from the last block is utilized for prediction in the downstream module.