ProtIR: Iterative Refinement between Retrievers and Predictors for Protein Function Annotation

TL;DR

Protein function annotation benefits from capturing inter-protein similarity, not just individual sequence or structure encodings. The authors introduce ProtIR, a variational EM framework that iteratively refines a function predictor and a neural structure retriever, leveraging pseudo-likelihood to model the joint distribution over protein functions. Empirically, retrievers can match predictors without massive pre-training, and ProtIR boosts predictor performance by about 10% on EC and GO tasks, achieving competitive results with PLM-based methods without large pre-training. This approach offers a practical, efficient pathway to high-accuracy function annotation and highlights the value of integrating similarity modeling into predictive models.

Abstract

Protein function annotation is an important yet challenging task in biology. Recent deep learning advancements show significant potential for accurate function prediction by learning from protein sequences and structures. Nevertheless, these predictor-based methods often overlook the modeling of protein similarity, an idea commonly employed in traditional approaches using sequence or structure retrieval tools. To fill this gap, we first study the effect of inter-protein similarity modeling by benchmarking retriever-based methods against predictors on protein function annotation tasks. Our results show that retrievers can match or outperform predictors without large-scale pre-training. Building on these insights, we introduce a novel variational pseudo-likelihood framework, ProtIR, designed to improve function predictors by incorporating inter-protein similarity modeling. This framework iteratively refines knowledge between a function predictor and retriever, thereby combining the strengths of both predictors and retrievers. ProtIR showcases around 10% improvement over vanilla predictor-based methods. Besides, it achieves performance on par with protein language model-based methods, yet without the need for massive pre-training, highlighting the efficacy of our framework. Code will be released upon acceptance.

Figures (7)

• Figure 1: High-level illustration of predictor- and retriever-based methods for protein function annotation.
• Figure 2: Overview of ProtIR. In the E-step and M-step, the neural predictor $\psi$ and retriever $\phi$ are trained, respectively, and their predictions iteratively refine each other. Before iterative refinement, the predictor $\psi$ and retriever $\phi$ are pre-trained on function prediction and fold classification, respectively.
• Figure 3: Fmax on four protein function annotation tasks under the 95% cutoff. The upper section (in blue) displays the results for methods based on predictors, whereas the lower section (in green) presents results for methods based on retrievers. The best methods in each section are highlighted in bold. Methods employing protein language models as predictors and retrievers are highlighted with hatches in dark blue and dark green, respectively. Detailed results are shown in Tab. \ref{['tab:all_result']}.
• Figure 4: Fmax on function annotation tasks vs. number of rounds in iterative refinement. Besides default ProtIR, we also depict curves without retriever pre-training on fold classification, highlighting the impact of injecting structural information.
• Figure 5: Training time of different methods at pre-training and fine-tuning stages on A100, where pre-training protein language models takes over 1K GPU hours.