MutaPLM: Protein Language Modeling for Mutation Explanation and Engineering

TL;DR

MutaPLM introduces a protein delta network that captures explicit protein mutation representations within a unified feature space, and a transfer learning pipeline with a chain-of-thought (CoT) strategy to harvest protein mutation knowledge from biomedical texts.

Abstract

Studying protein mutations within amino acid sequences holds tremendous significance in life sciences. Protein language models (PLMs) have demonstrated strong capabilities in broad biological applications. However, due to architectural design and lack of supervision, PLMs model mutations implicitly with evolutionary plausibility, which is not satisfactory to serve as explainable and engineerable tools in real-world studies. To address these issues, we present MutaPLM, a unified framework for interpreting and navigating protein mutations with protein language models. MutaPLM introduces a protein delta network that captures explicit protein mutation representations within a unified feature space, and a transfer learning pipeline with a chain-of-thought (CoT) strategy to harvest protein mutation knowledge from biomedical texts. We also construct MutaDescribe, the first large-scale protein mutation dataset with rich textual annotations, which provides cross-modal supervision signals. Through comprehensive experiments, we demonstrate that MutaPLM excels at providing human-understandable explanations for mutational effects and prioritizing novel mutations with desirable properties. Our code, model, and data are open-sourced at https://github.com/PharMolix/MutaPLM.

Figures (9)

• Figure 1: Model architecture of MutaPLM. (a) The encoding branch of the protein delta network. The delta encoder takes the subtraction of the PLM representations of the mutant and wild-type as inputs to generate $z_{\Delta}$. (b) The decoding branch of the protein delta network. The key components involve a delta decoder that reconstructs mutant features and two prediction heads deciding the position and amino acid of the mutation.
• Figure 2: Training pipeline of MutaPLM. (a) Workflow of pre-training on protein-related literature. We perform next token prediction for the encoding workflow and conditional masked language modeling for the decoding workflow. (b) Workflow of fine-tuning with chain-of-thought (CoT). We employ a two-round dialog that involves describing the functions of a wild-type protein, explaining the effects of its mutation, and predicting the mutation based on the mutational effects.
• Figure 3: Human-AI collaborative evaluation results for mutation explanation on the test sets of MutaDescribe. We show the number of accurate, relevant, opposite, and irrelevant predictions.
• Figure 4: Case study for a mutation from A (Alanine) to D (Aspartic) at the 205-th position of m7GpppX diphosphatase. MutaPLM provides accurate explanations and insights, while GPT-4 generates irrelevant results.
• Figure 5: Visualization of fitness scores for multi-round protein optimization. The curves indicate the average results, and the shaded regions indicate the standard deviation.