Semi-supervised variational autoencoder for cell feature extraction in multiplexed immunofluorescence images

TL;DR

The paper addresses robust cell feature extraction in multiplexed immunofluorescence (mIF) images without relying on exhaustive segmentation. It introduces a semi-supervised variational autoencoder with a fixed latent subspace $z'$ used for cell-phenotype supervision and a full latent $z$ for reconstruction, trained on $n=44{,}400$ cell patches across $1{,}093$ TMA cores, with $z \in \mathbb{R}^{9{,}216}$ and $z' \in \mathbb{R}^{1{,}152}$. The objective blends the evidence lower bound (ELBO) with a classification term, optimized as $L(\theta,\phi,\gamma) = - \sum_{(x,y)} [ ELBO(\theta,\phi;x) + \alpha \cdot \log l_{CE}(y, C_\gamma(\mu_\phi(x)')) ]$. The method achieves higher classification accuracy than baselines and indicates an optimal latent-subspace ratio of $z'/z = 1/8$, suggesting improved robustness of cell representations for tumor microenvironment analysis.

Abstract

Advancements in digital imaging technologies have sparked increased interest in using multiplexed immunofluorescence (mIF) images to visualise and identify the interactions between specific immunophenotypes with the tumour microenvironment at the cellular level. Current state-of-the-art multiplexed immunofluorescence image analysis pipelines depend on cell feature representations characterised by morphological and stain intensity-based metrics generated using simple statistical and machine learning-based tools. However, these methods are not capable of generating complex representations of cells. We propose a deep learning-based cell feature extraction model using a variational autoencoder with supervision using a latent subspace to extract cell features in mIF images. We perform cell phenotype classification using a cohort of more than 44,000 multiplexed immunofluorescence cell image patches extracted across 1,093 tissue microarray cores of breast cancer patients, to demonstrate the success of our model against current and alternative methods.

Figures (3)

• Figure 1: Proposed VAE framework for cell feature extraction in mIF image data.
• Figure 2: Example mIF image patches. (a) Expanded views of cells in a mIF TMA core. (b) Each channel of the mIF image captures the presence of a respective biomarker.
• Figure 3: Reconstruction and classification results. (a,b) Reconstruction results of our model accurately captures cellular features while minimising noise. (c) Confusion matrix shows our model is capable of classifying all cell phenotypes with high accuracy. Exhausted T-cells are more prone to be categorised as T-cells which may be due to the variation in staining strength of CD8+ and PD1+ biomarkers.