Fine-tuning Protein Language Models with Deep Mutational Scanning improves Variant Effect Prediction

TL;DR

The paper addresses the challenge of translating protein language model (PLM) predictions into accurate functional assessments of missense variants by leveraging Deep Mutational Scanning (DMS) data. It introduces a Lightweight Normalised Log-odds Ratio (NLR) head that normalises DMS scores across assays and enables end-to-end fine-tuning of PLMs, specifically adapting ESM-family models. Evaluations on held-out proteins and independent benchmarks such as ProteinGym and ClinVar show consistent improvements in correlation and pathogenicity classification, with larger gains for proteins that underperform in zero-shot predictions. The work demonstrates that integrating experimental DMS maps with PLMs is a scalable path to enhance variant effect prediction, while highlighting opportunities to extend the approach to MSA-based PLMs and larger DMS datasets.

Abstract

Protein Language Models (PLMs) have emerged as performant and scalable tools for predicting the functional impact and clinical significance of protein-coding variants, but they still lag experimental accuracy. Here, we present a novel fine-tuning approach to improve the performance of PLMs with experimental maps of variant effects from Deep Mutational Scanning (DMS) assays using a Normalised Log-odds Ratio (NLR) head. We find consistent improvements in a held-out protein test set, and on independent DMS and clinical variant annotation benchmarks from ProteinGym and ClinVar. These findings demonstrate that DMS is a promising source of sequence diversity and supervised training data for improving the performance of PLMs for variant effect prediction.

Figures (6)

• Figure 1: Methods overview. A) Preparation of normalised DMS functional scores from a subset of MaveDB experiments. The mean scores of synonymous and nonsense variants are used to create a common scale across assays and proteins. B) Fine-tuning pipeline for ESM-1v models using the Normalised Log-odds Ratio (NLR) head. C) Performance evaluation on two independent benchmarks: DMS assays from ProteinGym, and pathogenic and benign missense variants from ClinVar.
• Figure 2: Results after NLR fine-tuning of ESM-1v models across benchmarks. A) Performance in the five MaveDB test proteins. ProteinGym DMS assays and ClinVar pathogenic variants. B) Spearman correlation in MaveDB test proteins. Mean $\pm$ standard deviation (std) of 50 bootstrapped samples. C) Spearman correlation in ProteinGym DMS assays. Mean $\pm$ std of 50 bootstrapped samples. D) Per-protein auROC for ClinVar proteins with over 10 benign and 10 pathogenic variants.
• Figure S1: Normalisation of DMS functional scores. A) Distribution of DMS functional scores for missense, nonsense, and synonymous variants in MaveDB for the PTEN protein after score rescaling and normalisation, shown as density functions. B) Stacked histogram of DMS functional scores for all missense variants in the 30 proteins of the final normalised MaveDB dataset. C) Distributions of normalised DMS functional scores for each of the 30 proteins in the MaveDB dataset, shown as half-violin plots.
• Figure S2: Diagram of the fine-tuning ESM-1v architecture with Normalised Log-odds Ratio (NLR) head. (1) Training instance with a single amino acid swap and its DMS label. (2) Input token representation for the wild-type sequence. (3) ESM-1v pre-trained encoder blocks + Language Modelling head. (4) Fine-tuning blocks, including log-odds ratio matrix calculation and normalisation layers. (5) Output matrix with the chosen cell reflecting the predicted score for the input variant.
• Figure S3: A) DMS vs. predicted log-odds ratios for each variant in the MaveDB test set, stratified by gene. In blue are the scores from the zero-shot ESM-1v, while in red are the NLR fine-tuned scores. A linear fit is displayed together with the score distribution and the Spearman correlation is shown for each distribution. B) Predicted log-odds ratios from ESM-1v vs. NLR-finetuned ESM-1v, for each variant in the ClinVar benchmark. In blue and red are the benign and pathogenic variants, respectively.