Decoding Translation-Related Functional Sequences in 5'UTRs Using Interpretable Deep Learning Models

TL;DR

UTR-STCNet consistently outperforms state-of-the-art baselines in predicting mean ribosome load (MRL), a key proxy for translational efficiency, and recovers known functional elements such as upstream AUGs and Kozak motifs, highlighting its potential for mechanistic insight into translation regulation.

Abstract

Understanding how 5' untranslated regions (5'UTRs) regulate mRNA translation is critical for controlling protein expression and designing effective therapeutic mRNAs. While recent deep learning models have shown promise in predicting translational efficiency from 5'UTR sequences, most are constrained by fixed input lengths and limited interpretability. We introduce UTR-STCNet, a Transformer-based architecture for flexible and biologically grounded modeling of variable-length 5'UTRs. UTR-STCNet integrates a Saliency-Aware Token Clustering (SATC) module that iteratively aggregates nucleotide tokens into multi-scale, semantically meaningful units based on saliency scores. A Saliency-Guided Transformer (SGT) block then captures both local and distal regulatory dependencies using a lightweight attention mechanism. This combined architecture achieves efficient and interpretable modeling without input truncation or increased computational cost. Evaluated across three benchmark datasets, UTR-STCNet consistently outperforms state-of-the-art baselines in predicting mean ribosome load (MRL), a key proxy for translational efficiency. Moreover, the model recovers known functional elements such as upstream AUGs and Kozak motifs, highlighting its potential for mechanistic insight into translation regulation.

Figures (6)

• Figure 1: Overview of UTR-STCNet. The model takes variable-length 5'UTRs sequences as input to predict mean ribosome load (MRL). It combines a Saliency-Aware Token Clustering (SATC) module for selecting informative tokens with a Saliency-Guided Transformer (SGT) block for modeling regulatory interactions. The output representations are biologically interpretable, enabling recovery of known sequence elements.
• Figure 2: Prediction performance of UTR-STCNet on MRL. Evaluation was conducted on MPRA datasets with fixed-length sequences (left, n = 20,000) and variable-length sequences (right, n = 7,600). Performance was measured using the coefficient of determination ($\text{R}^2$) between observed and predicted MRL values on held-out test sets.
• Figure 3: Top enriched 3-mer motifs identified within high-saliency regions. High-saliency regions were first identified as stretches of consecutive nucleotides with elevated saliency scores. Within these regions, a 3-mer sliding window (step size = 3) was applied to extract all possible 3-mer motifs. A 3-mer was counted once per sequence if it appeared in a high-saliency region and exceeded a predefined score threshold. The top 15 3-mers with the highest frequency across all sequences are shown.
• Figure 4: Distribution of saliency scores across TG-related groups. Density plots show the distribution of token-level saliency scores for different TG-related groups in panels a and b.
• Figure 5: In silico mutagenesis reveals the functional impact of TG-related motifs on MRL predictions. a, Predicted MRL values after replacing the nucleotide preceding the 'TG' motif with adenine (A). Dark blue: mutated sequences; light blue: original experimental sequences. b, Predicted MRL values after replacing the 'TG' motif itself with non-TG variants. Dark blue: substituted sequences; light blue: original sequences.