Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis

TL;DR

WiKG introduces a dynamic directed graph representation for WSIs by learning head and tail embeddings for patches and constructing directed edge embeddings, enabling flexible neighbor interactions beyond fixed spatial graphs. A knowledge-aware attention mechanism aggregates neighbor information via head-tail-edge triplets, updating head features before a global pooling Readout produces a graph-level embedding for WSI classification. Evaluations on three TCGA datasets (ESCA, KIDNEY, LUNG) and an in-house FROZEN-LUNG set show state-of-the-art performance and strong generalization. The method is end-to-end trainable and code is released.

Abstract

Histopathological whole slide images (WSIs) classification has become a foundation task in medical microscopic imaging processing. Prevailing approaches involve learning WSIs as instance-bag representations, emphasizing significant instances but struggling to capture the interactions between instances. Additionally, conventional graph representation methods utilize explicit spatial positions to construct topological structures but restrict the flexible interaction capabilities between instances at arbitrary locations, particularly when spatially distant. In response, we propose a novel dynamic graph representation algorithm that conceptualizes WSIs as a form of the knowledge graph structure. Specifically, we dynamically construct neighbors and directed edge embeddings based on the head and tail relationships between instances. Then, we devise a knowledge-aware attention mechanism that can update the head node features by learning the joint attention score of each neighbor and edge. Finally, we obtain a graph-level embedding through the global pooling process of the updated head, serving as an implicit representation for the WSI classification. Our end-to-end graph representation learning approach has outperformed the state-of-the-art WSI analysis methods on three TCGA benchmark datasets and in-house test sets. Our code is available at https://github.com/WonderLandxD/WiKG.

Figures (7)

• Figure 1: Illustration of three different frameworks of instance-bag representation, conventional undirected graph representation, and the proposed dynamic graph representation.
• Figure 2: The framework of our proposed method for WSI analysis, including patch feature extraction, dynamic edge construction based on head and tail embeddings, graph representation learning, and the prediction of WSIs.
• Figure 3: Illustration of our proposed knowledge-aware attention mechanism, including aggregation between head, tail, and edge embeddings.
• Figure 4: Typing and staging results of AUC and Accuracy scores with different numbers of neighbor nodes on three TCGA datasets.
• Figure 5: Convergence curves of validation AUC and training time of each epoch in TCGA-ESCA.