SimpleFold: Folding Proteins is Simpler than You Think

TL;DR

SimpleFold shows that protein folding can be effectively tackled with a flow-matching generative model built entirely from general-purpose transformers, avoiding domain-specific modules like MSA, pair representations, or triangle updates. By scaling to 3B parameters and training on millions of distilled and experimental structures, it achieves competitive folding performance on CAMEO22 and CASP14 and excels at generating ensembles. The approach enables efficient inference on consumer hardware and provides a robust confidence signal via pLDDT, suggesting a scalable, practical alternative design space for protein structure prediction. The work also demonstrates strong ensemble capabilities and highlights the value of large-scale distilled-data training for single-sequence folding models.

Abstract

Protein folding models have achieved groundbreaking results typically via a combination of integrating domain knowledge into the architectural blocks and training pipelines. Nonetheless, given the success of generative models across different but related problems, it is natural to question whether these architectural designs are a necessary condition to build performant models. In this paper, we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term. We scale SimpleFold to 3B parameters and train it on approximately 9M distilled protein structures together with experimental PDB data. On standard folding benchmarks, SimpleFold-3B achieves competitive performance compared to state-of-the-art baselines, in addition SimpleFold demonstrates strong performance in ensemble prediction which is typically difficult for models trained via deterministic reconstruction objectives. Due to its general-purpose architecture, SimpleFold shows efficiency in deployment and inference on consumer-level hardware. SimpleFold challenges the reliance on complex domain-specific architectures designs in protein folding, opening up an alternative design space for future progress.

Figures (11)

• Figure 1: Example predictions of SimpleFold on targets (a) chain A of 7QSW (RubisCO large subunit) and (b) chain A of 8DAY (Dimethylallyltryptophan synthase 1), with ground truth shown in light aqua and prediction in deep teal. (c) Generated ensembles of target chain B of 6NDW (Flagellar hook protein FlgE) with SimpleFold finetuned on MD ensemble data. (d) Performance of SimpleFold on CASP14 with increasing model sizes from 100M to 3B. (e) Inference time of different sizes of SimpleFold on consumer level hardware, i.e., M2 Max 64GB Macbook Pro.
• Figure 2: Overview of SimpleFold's architecture built on general-purpose standard Transformer block with adaptive layers. Atom encoder, residue trunk, and atom decoder all share the same general-purposed building block. Our model circumvents the need for pair representations or triangular updates.
• Figure 3: (a) An example prediction of SimpleFold with pLDDT (color red to dark blue denote pLDDT low to high following visualization from chakravarty2022alphafold2). (b) & (c) Comparison of pLDDT and LDDT-$C_\alpha$.
• Figure 4: Scaling behavior of SimpleFold. Training Gflops vs. folding performance on GDT-TS and (b) TM-score. Training steps vs. folding performance on (c) GDT-TS and (d) TM-score. How data scale affects the performance (e) GDT-TS and (f) TM-score. All models are benchmarked on CAMEO22.
• Figure 5: Major neural network blocks of (a) Evoformer in AlphaFold2, and (b) Transformer with adaptive layer in SimpleFold.