Meta-Learning for Cross-Task Generalization in Protein Mutation Property Prediction

TL;DR

This work introduces a systematic meta-learning framework for cross-task generalization in protein mutation property prediction by applying Model-Agnostic Meta-Learning (MAML) to transformer-based sequence models. It couples MAML with an Enhanced Mutation Encoding that integrates mutation information directly into sequence context, addressing the limitation of mutations being treated as unknown tokens. Using a multi-source dataset comprising ProteinGym, FireProtDB, and SoluProtMutDB, the approach demonstrates superior cross-task and pooled performance across functional fitness, stability, and solubility prediction tasks, with substantial training-time savings. The results indicate notable improvements in accuracy (e.g., up to 94% NMSE improvement for solubility) and efficiency (up to ~65% faster training) over traditional fine-tuning, supporting the method’s practical value for industrial protein design where data are scarce. Overall, the combination of MAML and Enhanced Encoding provides a principled pathway to generalizable mutation-effect prediction across diverse experimental conditions and protein families.

Abstract

Protein mutations can have profound effects on biological function, making accurate prediction of property changes critical for drug discovery, protein engineering, and precision medicine. Current approaches rely on fine-tuning protein-specific transformers for individual datasets, but struggle with cross-dataset generalization due to heterogeneous experimental conditions and limited target domain data. We introduce two key innovations: (1) the first application of Model-Agnostic Meta-Learning (MAML) to protein mutation property prediction, and (2) a novel mutation encoding strategy using separator tokens to directly incorporate mutations into sequence context. We build upon transformer architectures integrating them with MAML to enable rapid adaptation to new tasks through minimal gradient steps rather than learning dataset-specific patterns. Our mutation encoding addresses the critical limitation where standard transformers treat mutation positions as unknown tokens, significantly degrading performance. Evaluation across three diverse protein mutation datasets (functional fitness, thermal stability, and solubility) demonstrates significant advantages over traditional fine-tuning. In cross-task evaluation, our meta-learning approach achieves 29% better accuracy for functional fitness with 65% less training time, and 94% better accuracy for solubility with 55% faster training. The framework maintains consistent training efficiency regardless of dataset size, making it particularly valuable for industrial applications and early-stage protein design where experimental data is limited. This work establishes a systematic application of meta-learning to protein mutation analysis and introduces an effective mutation encoding strategy, offering transformative methodology for cross-domain generalization in protein engineering.

Figures (3)

• Figure 1: Overview of the MAML framework applied to protein sequence modeling. The dataset is split into support and query sets for each sampled task $\tau_i$. The support set is used in the inner loop to compute the task-specific loss and adapt the model parameters $\theta_i$ to $\theta_i'$. The query set is then used in the outer loop to compute the meta-loss and update the original model parameters $\theta_i$. The updated parameters are used to predict protein properties by feeding the data through a forward network that integrates both the protein sequences and their mutation information.
• Figure 2: Distributions of protein property types in the integrated dataset: (a) Functional Fitness (FF) (b) Change in Gibbs Free Energy ($\Delta\Delta G$), (c) Change in Melting Temperature ($\Delta T_m$), and (d) Change in Solubility ($\Delta S$). Histograms are truncated at the 95th percentile for clarity; extreme outliers are excluded from visualization but retained in all analyses.
• Figure 3: Comparison of input representations for encoding protein mutations in transformer models. The top row shows the Standard Encoding, where the mutation is appended as a separate string (e.g., R9A). This format causes issues due to the mutation position being treated as an unknown token ([UNK]), resulting in loss of critical positional and contextual information. The bottom row illustrates our proposed Enhanced Encoding, where the mutation is integrated directly into the sequence using [SEP] tokens to delineate the wild-type residue, the mutated residue, and surrounding sequence context. This approach retains full token information and improves mutation interpretability by the model. It is important to note here that even though ProtBERT tokenizes per amino acid, it is not shown in the figure as such for better clarity.