Graph Neural Networks in Histopathology: Emerging Trends and Future Directions

TL;DR

This comprehensive review of graph neural networks in histopathology surveys GNNs in histopathology, discusses their applications, and explores emerging trends that pave the way for future advancements in the field.

Abstract

Histopathological analysis of Whole Slide Images (WSIs) has seen a surge in the utilization of deep learning methods, particularly Convolutional Neural Networks (CNNs). However, CNNs often fall short in capturing the intricate spatial dependencies inherent in WSIs. Graph Neural Networks (GNNs) present a promising alternative, adept at directly modeling pairwise interactions and effectively discerning the topological tissue and cellular structures within WSIs. Recognizing the pressing need for deep learning techniques that harness the topological structure of WSIs, the application of GNNs in histopathology has experienced rapid growth. In this comprehensive review, we survey GNNs in histopathology, discuss their applications, and explore emerging trends that pave the way for future advancements in the field. We begin by elucidating the fundamentals of GNNs and their potential applications in histopathology. Leveraging quantitative literature analysis, we identify four emerging trends: Hierarchical GNNs, Adaptive Graph Structure Learning, Multimodal GNNs, and Higher-order GNNs. Through an in-depth exploration of these trends, we offer insights into the evolving landscape of GNNs in histopathological analysis. Based on our findings, we propose future directions to propel the field forward. Our analysis serves to guide researchers and practitioners towards innovative approaches and methodologies, fostering advancements in histopathological analysis through the lens of graph neural networks.

Figures (8)

• Figure 1: Overview of the four emerging subtopics of GNNs in Histopathology, covered in this review: Hierarchical GNNs, Multimodal GNNS, Higher-order Graphs, and Adaptive Graph Structure Learning. mirabadi2024grasppati2020hactchen2020pathomicdi2022bigzhu2021deep
• Figure 2: Most widely used graph types in GNNs for histopathology. A) Cell Graph, B) Patch Graph, C) Tissue graph (based on superpixels, clustered superpixels, or a semantic segmentation mask. The superpixel image was acquired from bejnordi2015multi.)
• Figure 3: Most widely used graph construction techniques in GNNs for histopathology. A) Delaunay triangulation. B) K-NN with $k=3$, C) Distance threshold with threshold $t$, D) RAG with diagonal neighbors ($k=8$).
• Figure 4: Overview of a typical workflow of applying GNNs to histopathology whole slide images. A) First, preprocessing steps, such as slide quality thresholds and tissue segmentation (e.g., using Otsu thresholding) are applied. B) Then, if chosen for a patch graph approach, the WSI is divided into smaller image patches. C) When a cell graph approach is used, nuclei-segmentation algorithms are applied to acquire a mask of the nuclei in the WSI. D) For each acquired entity (patch, nucleus) features are extracted, typically using a pretrained CNN-model (e.g., ResNet) to acquire a feature matrix $X$. E) Using a graph construction strategy (e.g., k-NN), entities are connected to other entities to form a cell/patch graph, $G$. F) Now, this graph, along with its associated feature matrix, can be used as input for a GNN model which applies message passing operations to learn a representation and then produces an output depending on the prediction task. G) (Graph) explainability methods can be applied to the GNN model to acquire interpretable information on the model behaviour and its predictions.
• Figure 5: Cumulative frequency of publications on GNNs applied on histopathology, with different properties (e.g., Application, Graph type). For the types of message passing, graph types, graph constructors, and applications, only properties occurring in more than 4 papers were retained in the plot.