NACNet: A Histology Context-aware Transformer Graph Convolution Network for Predicting Treatment Response to Neoadjuvant Chemotherapy in Triple Negative Breast Cancer

TL;DR

It is suggested that NACNet holds strong potential for stratifying TNBC patients by NAC response, thereby helping to prevent overtreatment, improve patient quality of life, reduce treatment cost, and enhance clinical outcomes, marking an important advancement toward personalized breast cancer treatment.

Abstract

Neoadjuvant chemotherapy (NAC) response prediction for triple negative breast cancer (TNBC) patients is a challenging task clinically as it requires understanding complex histology interactions within the tumor microenvironment (TME). Digital whole slide images (WSIs) capture detailed tissue information, but their giga-pixel size necessitates computational methods based on multiple instance learning, which typically analyze small, isolated image tiles without the spatial context of the TME. To address this limitation and incorporate TME spatial histology interactions in predicting NAC response for TNBC patients, we developed a histology context-aware transformer graph convolution network (NACNet). Our deep learning method identifies the histopathological labels on individual image tiles from WSIs, constructs a spatial TME graph, and represents each node with features derived from tissue texture and social network analysis. It predicts NAC response using a transformer graph convolution network model enhanced with graph isomorphism network layers. We evaluate our method with WSIs of a cohort of TNBC patient (N=105) and compared its performance with multiple state-of-the-art machine learning and deep learning models, including both graph and non-graph approaches. Our NACNet achieves 90.0% accuracy, 96.0% sensitivity, 88.0% specificity, and an AUC of 0.82, through eight-fold cross-validation, outperforming baseline models. These comprehensive experimental results suggest that NACNet holds strong potential for stratifying TNBC patients by NAC response, thereby helping to prevent overtreatment, improve patient quality of life, reduce treatment cost, and enhance clinical outcomes, marking an important advancement toward personalized breast cancer treatment.

Figures (11)

• Figure 1: Overview of the NACNet architecture. Each WSI is partitioned into non-overlapping image tiles of size $150 \times 150$ pixels.A pre-trained convolutional model for image tile classification identifies all image tile classes, resulting in a tile-level histology label map where each pixel represents the histology label of an image tile. A sliding window moves over each histology label map and defines neighbors that share the same histology label as a WSI graph node. Both graph-based SNA and image texture features are used to represent the graph nodes. Nodes within the distance $\epsilon$ are connected with an edge. The resulting $\epsilon$-neighborhood graph is provided to a transformer-based GCN with GIN layers. This architecture incorporates the TME spatial information, improving the prediction power.
• Figure 2: Feature extration from WSI spatial TME graph. (a) Each WSI is partitioned into image tiles. A training set of slides are annotated with one of 12 histology classes. A VGG16 model VGG16 trained on these annotated slides predict the histology labels for the rest of the titles. All labeled tiles are then combined to produce the histology map for each WSI. (b) Within the histology map, a large cluster of tiles ($n>\eta$) sharing the same histology label is defined as a TME graph node. Any pair of nodes within distance $\epsilon$ are connected by an edges. Multiple SNA features are derived from each graph node (e.g., degree, betweenness, page rank, and closeness). (c) An autoencoder connected to a VGG flatten layer is used to find image embeddings of graph nodes. The node size, tissue texture and SNA features are integrated to enable a context-aware graph characterization.
• Figure 3: WSI-derived spatial TME graph. We present representative WSIs (top), the corresponding histology maps (middle), and the resulting WSI TME graph node distributions (bottom). The histology labels of image tiles of size $150 \times 150$ are classified and assembled to construct the histology map. In total, 12 histology labels are color coded, including hemorrhage, immune cells, carcinoma in situ (CIS), MVD, mucinous changes, necrosis, PGCC, stroma, tumor, adipose tissue, muscle tissue, and apocrine change.
• Figure 4: ROC curves of NACNet models in the ablation study. The ROC curves for the NACNet methods are generated for each cross-validation fold. $\ast$ denotes models without the GIN layer in the network.
• Figure 5: The ROC curves of methods for comparison. The ROC curves of (a) deep learning methods and (b) traditional machine learning methods.