Multi-modal Representation Learning Enables Accurate Protein Function Prediction in Low-Data Setting

TL;DR

The paper tackles protein function prediction in data-scarce settings by introducing HOPER, a multimodal framework that fuses protein sequences, biomedical text, and PPI networks through autoencoder-based representations and transfer learning. It systematically evaluates sequence-, text-, and PPI-based representations, then explores several multimodal fusion strategies, including SimpleAE, MultiModalAE, and TransferAE, using the PROBE benchmark across GO categories $MF$, $BP$, and $CC$. While text-based TF-IDF_PCA often performs strongest in low-data scenarios, transfer learning from multimodal representations (TransferAE) can improve sequence-only predictions, and a case study on LUAD immune-escape proteins demonstrates practical utility with 121 predictions and literature-backed targets. Overall, the results support multimodal representation learning as an effective approach to overcome data limitations in protein function prediction and suggest avenues for further improvement with advanced fusion methods and graph-based models.

Abstract

In this study, we propose HOPER (HOlistic ProtEin Representation), a novel multimodal learning framework designed to enhance protein function prediction (PFP) in low-data settings. The challenge of predicting protein functions is compounded by the limited availability of labeled data. Traditional machine learning models already struggle in such cases, and while deep learning models excel with abundant data, they also face difficulties when data is scarce. HOPER addresses this issue by integrating three distinct modalities - protein sequences, biomedical text, and protein-protein interaction (PPI) networks - to create a comprehensive protein representation. The model utilizes autoencoders to generate holistic embeddings, which are then employed for PFP tasks using transfer learning. HOPER outperforms existing methods on a benchmark dataset across all Gene Ontology categories, i.e., molecular function, biological process, and cellular component. Additionally, we demonstrate its practical utility by identifying new immune-escape proteins in lung adenocarcinoma, offering insights into potential therapeutic targets. Our results highlight the effectiveness of multimodal representation learning for overcoming data limitations in biological research, potentially enabling more accurate and scalable protein function prediction. HOPER source code and datasets are available at https://github.com/kansil/HOPER

Figures (6)

• Figure 1: The overview of the study. We first generated protein representations (embeddings) independently using three different modalities (i.e., protein sequence, protein-protein interaction, and text). Then, we benchmarked them to find the best-performing representation model for each modality in the context of protein function prediction in low training data setting. After that, we constructed multimodal learning models that take representations of independent modalities as input and produce a holistic embedding by leveraging their relationships. Finally, as a use-case study, we predicted new tumor immune-escape proteins in the lung adenocarcinoma disease using our model and discussed the results.
• Figure 2: Ontology-based protein function prediction benchmark results. Heat maps indicating the clustered performance results (weighted F1-scores) of protein representation methods in ontology-based PFP benchmark in terms of GO categories of (a) molecular function, (b) biological process and (c) cellular component. The colours indicate groups of models (yellow, rule-based annotation methods; green, classical representations; blue, small-scale learned representations; red, large-scale learned representations).
• Figure 3: Protein function prediction performances of PPI representations based on mean F1-weighted scores. Multiple models that utilize Node2Vec and HOPE algorithms are shown with different hyperparameter selections (p and q for Node2Vec and beta for HOPE). Tests were conducted on the low data setting (i.e., MF, BP, and CC GO terms with low number of annotated proteins) for; a MF; b BP; and c CC terms.
• Figure 4: Protein function prediction performances of text-based representations based on mean F1-weighted scores. Multiple models that utilize TF-IDF-PCA, BioSentVec, and BioBERT algorithms are shown with different hyperparameter and data source selections. Tests were conducted on the low data setting (i.e., MF, BP, and CC GO terms with low number of annotated proteins) for; a MF; b BP; and c CC terms.
• Figure 5: Construction of multimodal representations using autoencoder models; a simple autoencoder (SimpleAE), b multimodal autoencoder (MultiModelAE), and c transfer learning-based sequence-input multimodal autoencoder (TransferAE).