CryoLVM: Self-supervised Learning from Cryo-EM Density Maps with Large Vision Models

TL;DR

CryoLVM introduces a JEPA-based self-supervised foundation model for cryo-EM density maps, leveraging a SCUNet backbone to learn semantic, transferable structural representations from 3D patches. It augments JEPA with an EMA-updated Target Encoder and a novel histogram KL loss to align predicted and ground-truth density distributions, accelerating convergence and improving downstream fine-tuning. The model is pretrained on a large corpus of experimental maps and finetuned for three tasks—density map sharpening, super-resolution, and missing wedge restoration—where it achieves state-of-the-art results across quantitative metrics and real-world maps. This work demonstrates the feasibility and value of a unified foundation-model approach in cryo-EM, enabling more scalable and generalizable AI-assisted structural biology workflows.

Abstract

Cryo-electron microscopy (cryo-EM) has revolutionized structural biology by enabling near-atomic-level visualization of biomolecular assemblies. However, the exponential growth in cryo-EM data throughput and complexity, coupled with diverse downstream analytical tasks, necessitates unified computational frameworks that transcend current task-specific deep learning approaches with limited scalability and generalizability. We present CryoLVM, a foundation model that learns rich structural representations from experimental density maps with resolved structures by leveraging the Joint-Embedding Predictive Architecture (JEPA) integrated with SCUNet-based backbone, which can be rapidly adapted to various downstream tasks. We further introduce a novel histogram-based distribution alignment loss that accelerates convergence and enhances fine-tuning performance. We demonstrate CryoLVM's effectiveness across three critical cryo-EM tasks: density map sharpening, density map super-resolution, and missing wedge restoration. Our method consistently outperforms state-of-the-art baselines across multiple density map quality metrics, confirming its potential as a versatile model for a wide spectrum of cryo-EM applications.

Figures (18)

• Figure 1: CryoLVM framework. During pretraining, CryoLVM leverages JEPA to learn rich structural representations from cryo-EM density maps. Input density maps are split into non-overlapping 3D patches and random sets of 3D patches are masked to produce context and target patches. The Target Predictor receives context embeddings from the Context Encoder along with positional information of masked target patches, and predicts the corresponding Target Encoder outputs. A regression loss is applied to masked tokens, encouraging alignment between predicted and target voxel embeddings. The weights of Target Encoder are updated via an exponential moving average (EMA) of the Context Encoder weights. Following pretraining, we evaluate the model through fine-tuning on three downstream cryo-EM tasks.
• Figure 2: CryoLVM architecture. The Context Encoder and Target Encoder use hierarchical swin-conv (SC) blocks for multi-scale feature extraction, with outputs converted to patch embeddings and combined with 3D sinusoidal positional encodings. The Target Predictor employs transformer blocks to predict target representations. For downstream tasks, task-specific decoder with upsampling SC blocks are jointly fine-tuned with the pretrained encoder.
• Figure 3: Comparative evaluation of density map sharpening performance across baseline and proposed methods. Cross-correlation metrics ($\rm CC_{box}$, $\rm CC_{mask}$, $\rm CC_{peaks}$) were calculated via phenix.map_model_cc afonine2018new, which quantify the agreement between density maps and their associated atomic models over different spatial regions. Q-score was computed using Chimera.MapQ pintilie2020measurementpintilie2025q, providing an independent assessment of map quality based on local atom-to-density correlation and atomic resolvability.
• Figure 3: Performance of different methods in missing wedge restoration task. The score is computed between two half maps (predicted map and ground truth map). Additional results are in Appendix \ref{['app:add_results_mw']}.
• Figure 4: Comparison of map super-resolution performance between different methods. FSC-based metrics computed using phenix.mtriage afonine2018new. Local resolution estimates obtained via CryoRes dai2023cryores; global resolution represents the average of voxel-wise predictions.