Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences

Azwad Tamir, Jiann-Shiun Yuan

TL;DR

ProtGO tackles GO term annotation from full-length protein sequences using a fusion of three transformer blocks, each specialized for BP, MF, or CC terms, and leveraging selectively finetuned ProtBert pretrained weights to improve efficiency. The model is trained on Swiss-Prot-derived data with random and clustered splits, and evaluated against Proteinfer baselines, achieving state-of-the-art accuracy and robust performance across sequence lengths. The work demonstrates strong ROC AUC performance (MF typically highest) and presents evidence that ProtGO generalizes to structurally diverse sequences better than baselines, especially in the more challenging clustered split. These results suggest ProtGO as a practical, scalable solution for large-scale protein annotation tasks.

Abstract

Recent developments in next generation sequencing technology have led to the creation of extensive, open-source protein databases consisting of hundreds of millions of sequences. To render these sequences applicable in biomedical applications, they must be meticulously annotated by wet lab testing or extracting them from existing literature. Over the last few years, researchers have developed numerous automatic annotation systems, particularly deep learning models based on machine learning and artificial intelligence, to address this issue. In this work, we propose a transformer-based fusion model capable of predicting Gene Ontology (GO) terms from full-scale protein sequences, achieving state-of-the-art accuracy compared to other contemporary machine learning annotation systems. The approach performs particularly well on clustered split datasets, which comprise training and testing samples originating from distinct distributions that are structurally diverse. This demonstrates that the model is able to understand both short and long term dependencies within the enzyme's structure and can precisely identify the motifs associated with the various GO terms. Furthermore, the technique is lightweight and less computationally expensive compared to the benchmark methods, while at the same time not unaffected by sequence length, rendering it appropriate for diverse applications with varying sequence lengths.

ProtGO: A Transformer based Fusion Model for accurately predicting Gene Ontology (GO) Terms from full scale Protein Sequences

TL;DR

ProtGO tackles GO term annotation from full-length protein sequences using a fusion of three transformer blocks, each specialized for BP, MF, or CC terms, and leveraging selectively finetuned ProtBert pretrained weights to improve efficiency. The model is trained on Swiss-Prot-derived data with random and clustered splits, and evaluated against Proteinfer baselines, achieving state-of-the-art accuracy and robust performance across sequence lengths. The work demonstrates strong ROC AUC performance (MF typically highest) and presents evidence that ProtGO generalizes to structurally diverse sequences better than baselines, especially in the more challenging clustered split. These results suggest ProtGO as a practical, scalable solution for large-scale protein annotation tasks.

Abstract

Recent developments in next generation sequencing technology have led to the creation of extensive, open-source protein databases consisting of hundreds of millions of sequences. To render these sequences applicable in biomedical applications, they must be meticulously annotated by wet lab testing or extracting them from existing literature. Over the last few years, researchers have developed numerous automatic annotation systems, particularly deep learning models based on machine learning and artificial intelligence, to address this issue. In this work, we propose a transformer-based fusion model capable of predicting Gene Ontology (GO) terms from full-scale protein sequences, achieving state-of-the-art accuracy compared to other contemporary machine learning annotation systems. The approach performs particularly well on clustered split datasets, which comprise training and testing samples originating from distinct distributions that are structurally diverse. This demonstrates that the model is able to understand both short and long term dependencies within the enzyme's structure and can precisely identify the motifs associated with the various GO terms. Furthermore, the technique is lightweight and less computationally expensive compared to the benchmark methods, while at the same time not unaffected by sequence length, rendering it appropriate for diverse applications with varying sequence lengths.

Paper Structure

This paper contains 10 sections, 1 equation, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Model
  3. GO Terms
  4. Dataset
  5. Model Architecture
  6. Training and Evaluation
  7. Results
  8. ROC Curves
  9. Sequence Length Analysis
  10. Conclusion

Figures (3)

  • Figure 1: Block diagram illustrating the detailed architecture of the proposed model of ProtGO.
  • Figure 2: Left: Averaged ROC curve of the ProtGO model for the random split dataset. Right: Averaged ROC curve of the ProtGO model for the Clustered split dataset.
  • Figure 3: Left: Frequency distribution of the input protein sequence lengths in the dataset. Right: Variability of ProtGO Accuracy for the different GO aspects on the input sequence length.