Open-Source Protein Language Models for Function Prediction and Protein Design
Shivasankaran Vanaja Pandi, Bharath Ramsundar
TL;DR
Open-Source Protein Language Models for Function Prediction and Protein Design addresses the barrier to entry for large PLMs by embedding ProtBERT within the Open-Source DeepChem framework. The authors demonstrate competitive performance on several protein-prediction tasks using embeddings from a pretrained model trained on ~1M sequences, and they explore latent-space manipulation to generate plastic-degrading enzyme candidates. They validate generated structures with AlphaFold, providing a proof-of-concept for environment-focused protein design under resource constraints. The work offers a reusable, accessible platform to accelerate protein-function prediction and design in synthetic biology and sustainability research.
Abstract
Protein language models (PLMs) have shown promise in improving the understanding of protein sequences, contributing to advances in areas such as function prediction and protein engineering. However, training these models from scratch requires significant computational resources, limiting their accessibility. To address this, we integrate a PLM into DeepChem, an open-source framework for computational biology and chemistry, to provide a more accessible platform for protein-related tasks. We evaluate the performance of the integrated model on various protein prediction tasks, showing that it achieves reasonable results across benchmarks. Additionally, we present an exploration of generating plastic-degrading enzyme candidates using the model's embeddings and latent space manipulation techniques. While the results suggest that further refinement is needed, this approach provides a foundation for future work in enzyme design. This study aims to facilitate the use of PLMs in research fields like synthetic biology and environmental sustainability, even for those with limited computational resources.