Prot42: a Novel Family of Protein Language Models for Target-aware Protein Binder Generation

TL;DR

The paper tackles the bottleneck of protein binder design by moving beyond structure-dependent methods to a long-context, decoder-only protein language model approach. Prot42, a pair of autoregressive pLMs with up to 1.1B parameters, is trained on vast unlabeled sequences and extended to a context window of $8{,}192$ residues, enabling modeling of large, multidomain proteins. It demonstrates two main applications: designing high-affinity protein binders from target sequences alone and generating sequence-specific DNA-binding proteins via a multimodal, DNA-conditioned design framework; it achieves strong demonstrations on PEER benchmarks and shows compelling, sub-nanomolar predicted $K_d$ binders for several targets, notably IL-7R$\alpha$ and PD-L1. The work suggests a scalable, structure-free path for rapid protein engineering, with practical impact in therapeutics and synthetic biology, and sets the stage for experimental validation of Prot42-generated binders.

Abstract

Unlocking the next generation of biotechnology and therapeutic innovation demands overcoming the inherent complexity and resource-intensity of conventional protein engineering methods. Recent GenAI-powered computational techniques often rely on the availability of the target protein's 3D structures and specific binding sites to generate high-affinity binders, constraints exhibited by models such as AlphaProteo and RFdiffusion. In this work, we explore the use of Protein Language Models (pLMs) for high-affinity binder generation. We introduce Prot42, a novel family of Protein Language Models (pLMs) pretrained on vast amounts of unlabeled protein sequences. By capturing deep evolutionary, structural, and functional insights through an advanced auto-regressive, decoder-only architecture inspired by breakthroughs in natural language processing, Prot42 dramatically expands the capabilities of computational protein design based on language only. Remarkably, our models handle sequences up to 8,192 amino acids, significantly surpassing standard limitations and enabling precise modeling of large proteins and complex multi-domain sequences. Demonstrating powerful practical applications, Prot42 excels in generating high-affinity protein binders and sequence-specific DNA-binding proteins. Our innovative models are publicly available, offering the scientific community an efficient and precise computational toolkit for rapid protein engineering.

Figures (8)

• Figure 2: Validation Perplexity (PPL) of Prot42-L models with different context length on the validation dataset. The input sequence lengths are varied from 1k to 8k.
• Figure 3: t-SNE visualization of Prot42-L protein embeddings across 10 subcellular localization compartments. Proteins cluster based on their localization as captured by Prot42-L embeddings.
• Figure 4: Illustration of the protein binding sequence generation process. Top: Target protein Vascular endothelial growth factor VEGF-A (PDB ID 1bj1) in complex with neutralizing antibody. Bottom: Multiple views and representations of a binding protein (in blue) generated by our Prot42 model for Chain V of VEGF-A. The model conditions on the target sequence $\mathbf{X}$ (VEGF-A), followed by $\text{SEP}$ token and initial methionine residue, generating binding sequence $\hat{\mathbf{Y}}$ autoregressively using $p(\mathbf{y} | \mathbf{x})$. The generated binder demonstrates a $K_d$ of 4.2nM, showcasing the model's capability to design proteins with specific binding properties.
• Figure 5: Generated binder example to specific DNA Chains C (ACCTGACGCGA) and D (TTCGCGTCAGG) in comparison with designed DNA binding protein (PDB ID 8TAC).
• Figure 6: Examples of generated protein binders (green molecular surfaces - structures in different colors) to target protein sequences provided in the PDB IDs.