Prior Knowledge Injection into Deep Learning Models Predicting Gene Expression from Whole Slide Images

TL;DR

We address the problem of predicting gene expression from Whole Slide Images as a cost-effective surrogate for RNA-Seq, which can be slow and expensive in clinical workflows. The authors introduce a model-agnostic prior knowledge (PK) injection framework that converts gene-gene co-expression data into gene embeddings via Nonnegative Matrix Factorization and then injects these embeddings into the gene-expression predictor with a tunable weight $\lambda$, forming the PK-augmented predictor $g_{pk} = (1-\lambda) w A^T + \lambda w G^T + b$. They evaluate across six model architectures, three PK sources (External, Internal, Combined), two feature extractors (CTrans, UNI) and three aggregators, reporting an average gain of $983$ significantly predicted genes on TCGA-BRCA across 18 experiments and successful generalization to CPTAC-BRCA in 14 of 18 cases, with UNI-based PK often providing the best robustness. The work demonstrates that injecting prior knowledge can improve predictive accuracy and robustness of WSI-based molecular profiling, potentially accelerating integration of transcriptomic markers into oncology workflows.

Abstract

Cancer diagnosis and prognosis primarily depend on clinical parameters such as age and tumor grade, and are increasingly complemented by molecular data, such as gene expression, from tumor sequencing. However, sequencing is costly and delays oncology workflows. Recent advances in Deep Learning allow to predict molecular information from morphological features within Whole Slide Images (WSIs), offering a cost-effective proxy of the molecular markers. While promising, current methods lack the robustness to fully replace direct sequencing. Here we aim to improve existing methods by introducing a model-agnostic framework that allows to inject prior knowledge on gene-gene interactions into Deep Learning architectures, thereby increasing accuracy and robustness. We design the framework to be generic and flexibly adaptable to a wide range of architectures. In a case study on breast cancer, our strategy leads to an average increase of 983 significant genes (out of 25,761) across all 18 experiments, with 14 generalizing to an increase on an independent dataset. Our findings reveal a high potential for injection of prior knowledge to increase gene expression prediction performance from WSIs across a wide range of architectures.

Figures (3)

• Figure 1: High level overview. We inject prior knowledge, here indicating that genes $g_1$ and $g_3$ are strongly correlated with $g_2$, into the gene expression Predictor. This guides the Predictor towards extracting correlated gene expression predictions from the WSI (e.g., for high $g_2$ expression, the expressions of $g_1$ and $g_3$ are encouraged to also be high).
• Figure 2: General workflow. First, the WSI is processed by an Encoder, which extracts patches, their corresponding features, and aggregates them into a single WSI embedding. Then, a Predictor transforms the embedding into gene expression predictions.
• Figure 3: Overview of our framework. First, we transform prior knowledge (gene-gene interaction network) into gene embeddings $G$ using a representation learning technique. The Encoder transforms the WSI into an embedding $\textbf{w}$. We then inject the gene embeddings into the Predictor by linearly transforming $\textbf{w}$ into gene predictions $\mathbf{g_{pk}}$ using a weighted sum of the linear predictor layer $A$, and the gene embeddings $G$. Hyperparameter $\lambda$ controls the effect of prior knowledge.