Machine learning identification of maternal inflammatory response and histologic choroamnionitis from placental membrane whole slide images

TL;DR

This work uses Multiple Instance Learning framework with 3 feature extractors and 2 histopathology foundation models, UNI and Phikon, to investigate predictability of MIR stage from histopathology WSIs, and finds that the pathology foundation models are both able to achieve higher performance with balanced accuracy and $\kappa$, compared to ImageNet-based feature extractor (EfficientNet-v2s).

Abstract

The placenta forms a critical barrier to infection through pregnancy, labor and, delivery. Inflammatory processes in the placenta have short-term, and long-term consequences for offspring health. Digital pathology and machine learning can play an important role in understanding placental inflammation, and there have been very few investigations into methods for predicting and understanding Maternal Inflammatory Response (MIR). This work intends to investigate the potential of using machine learning to understand MIR based on whole slide images (WSI), and establish early benchmarks. To that end, we use Multiple Instance Learning framework with 3 feature extractors: ImageNet-based EfficientNet-v2s, and 2 histopathology foundation models, UNI and Phikon to investigate predictability of MIR stage from histopathology WSIs. We also interpret predictions from these models using the learned attention maps from these models. We also use the MIL framework for predicting white blood cells count (WBC) and maximum fever temperature ($T_{max}$). Attention-based MIL models are able to classify MIR with a balanced accuracy of up to 88.5% with a Cohen's Kappa ($κ$) of up to 0.772. Furthermore, we found that the pathology foundation models (UNI and Phikon) are both able to achieve higher performance with balanced accuracy and $κ$, compared to ImageNet-based feature extractor (EfficientNet-v2s). For WBC and $T_{max}$ prediction, we found mild correlation between actual values and those predicted from histopathology WSIs. We used MIL framework for predicting MIR stage from WSIs, and compared effectiveness of foundation models as feature extractors, with that of an ImageNet-based model. We further investigated model failure cases and found them to be either edge cases prone to interobserver variability, examples of pathologist's overreach, or mislabeled due to processing errors.

Figures (8)

• Figure 1: (A) Model architecture for MIR stage classification. Each MIR class has a corresponding set of attention weights for the patch embeddings lu2021data. The aggregated embeddings for each class are processed by fully connected classification layers followed by a softmax layer to convert to class probabilities. The class with highest probability is the model prediction. Attention maps are visualized by coloring each patch based on corresponding attention weight. (B) Model architecture for white blood cell count (WBC) and maternal highest temperature ($T_{max}$) prediction
• Figure 2: Test set confusion matrices for EfficientNet (top), UNI (bottom-left), Phikon (bottom-right).
• Figure 3: t-SNE space of aggregated features from Phikon in the test set. Colors represent the true class. MIR0 is shown in dark blue, MIR1 in light blue, MIR2 in light red, and MIR3 in dark red.
• Figure 4: t-SNE embeddings of top-1 attended patches for attention branch 1 (See model architecture in Fig. \ref{['FIG:fig1']}), from each slide in the test set. Image patches corresponding to randomly sampled patches are shown with patching outlines highlighting the original diagnosis (MIR Stage) of corresponding WSIs. Blue patches are largely normal placenta or some neutrophil infiltration. The red patches show stroma with greater neutrophil infiltration. All the embeddings are shown in the inset.
• Figure 5: An example of a false positive. (A) WSI, (B) Attention Heatmap, (C) Top-10 attention patches from attention branch 1 (See model architecture in Fig. \ref{['FIG:fig1']}). This is a case with severe MIR1 with decidual necrosis, but does not meet criteria for MIR2.