Sparsity is All You Need: Rethinking Biological Pathway-Informed Approaches in Deep Learning

TL;DR

A comprehensive analysis of all relevant pathway-based neural network models for predictive tasks, critically evaluating each study's contributions and proposing a methodology that can be applied to different domains and can serve as a robust benchmark for systematically comparing novel pathway-informed models against their randomized counterparts.

Abstract

Biologically-informed neural networks typically leverage pathway annotations to enhance performance in biomedical applications. We hypothesized that the benefits of pathway integration does not arise from its biological relevance, but rather from the sparsity it introduces. We conducted a comprehensive analysis of all relevant pathway-based neural network models for predictive tasks, critically evaluating each study's contributions. From this review, we curated a subset of methods for which the source code was publicly available. The comparison of the biologically informed state-of-the-art deep learning models and their randomized counterparts showed that models based on randomized information performed equally well as biologically informed ones across different metrics and datasets. Notably, in 3 out of the 15 analyzed models, the randomized versions even outperformed their biologically informed counterparts. Moreover, pathway-informed models did not show any clear advantage in interpretability, as randomized models were still able to identify relevant disease biomarkers despite lacking explicit pathway information. Our findings suggest that pathway annotations may be too noisy or inadequately explored by current methods. Therefore, we propose a methodology that can be applied to different domains and can serve as a robust benchmark for systematically comparing novel pathway-informed models against their randomized counterparts. This approach enables researchers to rigorously determine whether observed performance improvements can be attributed to biological insights.

Figures (9)

• Figure 1: Schematic representation of pathway integration approaches in neural networks for omics data and their relative randomization. Pathway information can be incorporated in two ways (Panels a and c): (a) A neural network utilizing pathway information by enforcing structured connections, introducing sparsity in the model. (b) A randomized counterpart where connections are introduced without explicit pathway constraints allows for an alternative exploration of the data structure. (c) A data transformation strategy that incorporates pathway information to convert tabular omics data into graphs or images. (d) A randomized data transformation approach that generates graphs or images through a randomization procedure rather than predefined pathway structures.
• Figure 2: Circular bar plots summarizing characteristics of deep learning models that integrate pathway information. The plots show distributions for (a), Data Types used, (b), Year of Publication, (c), Pathway Database sources, (d), Model Architectures (FFNN-MLP: Feed-Forward Neural Network - Multi-Layer Perceptron, GNN: Graph Neural Network, CNN: Convolutional Neural Network, AE: Autoencoders), and (e), Prediction Tasks. Each segment’s length corresponds to the count of models within each category.
• Figure 3: Model performance comparison across Accuracy, AUC, C-Index, F1 Macro, and R-Square metrics using violin plots. Models are grouped as Pathway-Informed (pink) and Randomized (green). The width reflects the distribution of scores, with central lines for median values and box plots indicating interquartile ranges. Models for which the performance of the randomized version is significantly better than the pathway-informed version are bolded in the x-axis labels. The results for the MPVNN and DeepKEGG models represent average outcomes across different tumor types considered (detailed findings for each specific tumor type are provided in the Additional Information).
• Figure 4: Impact of sparsity on model performance. Optimal Sparsity Level: The green boxplots represent the performance (measured as Accuracy or AUC) of each model—BINN, DeepKEGG, PASNet, PathCNN and PINNet—across varying sparsity levels (60% to 99%). The pink boxplots indicate performance at the sparsity level induced by pathway information. For DeepKEGG, the pink boxplots are repeated, as the pathway-induced sparsity level varies across omics, ranging from 63.7% for miRNAs to 98.9% for mRNAs. In general, boxplots illustrate the distribution of performance across runs, while violin plots provide density estimates. The dashed pink line marks the performance of the pathway-derived sparsity model. Pathway-Induced sparsity levels for all models are reported in Tables \ref{['tab:models_sparsity_table']}, \ref{['tab:deepkegg']} and \ref{['tab:models_pathway_sparsity_table']} in the Additional Information.
• Figure 5: Hypothetical causes for the alignment in performance between pathway-informed and randomized models. Despite integrating biological knowledge, randomized models often perform comparably or better with respect to models incorporating pathway information. This figure summarizes several hypothetical factors that may contribute to explain this phenomenon.