Protein structure generation via folding diffusion

TL;DR

This work introduces FoldingDiff, a diffusion-based model that directly generates protein backbone structures by diffusing over a six-angle internal representation per residue, removing the need for equivariant 3D coordinates. Using a vanilla transformer as a denoiser and a wrapped-normal forward process, the model learns to produce realistic angle configurations that recapitulate native angle distributions and Ramachandran patterns, including chirality. Evaluation shows that approximately 22.7% of generated backbones are designable (scTM ≥ 0.5) with consistent diversity across replicates, and a strong reconstruction accuracy (>0.95 TM-score on average) indicates faithful geometry. The work provides substantial open-source resources and highlights a scalable, biologically-inspired path for de novo protein design, while outlining avenues to extend length, complexity, and functional integration.

Abstract

The ability to computationally generate novel yet physically foldable protein structures could lead to new biological discoveries and new treatments targeting yet incurable diseases. Despite recent advances in protein structure prediction, directly generating diverse, novel protein structures from neural networks remains difficult. In this work, we present a new diffusion-based generative model that designs protein backbone structures via a procedure that mirrors the native folding process. We describe protein backbone structure as a series of consecutive angles capturing the relative orientation of the constituent amino acid residues, and generate new structures by denoising from a random, unfolded state towards a stable folded structure. Not only does this mirror how proteins biologically twist into energetically favorable conformations, the inherent shift and rotational invariance of this representation crucially alleviates the need for complex equivariant networks. We train a denoising diffusion probabilistic model with a simple transformer backbone and demonstrate that our resulting model unconditionally generates highly realistic protein structures with complexity and structural patterns akin to those of naturally-occurring proteins. As a useful resource, we release the first open-source codebase and trained models for protein structure diffusion.

Figures (22)

• Figure 1: We perform diffusion on six angles as illustrated in the schematic in the bottom center (also defined in Table \ref{['tab:angles-definition']}). Three of these are dihedral torsion angles (orange), and three are bond angles (green). We start with an experimentally observed backbone described by angles $x_0$ and iteratively add Gaussian noise via the forward noising process $q$ until the angles are indistinguishable from a wrapped Gaussian at $x_T$. We use these examples to learn the "reverse" denoising process $p_\xi$.
• Figure 2: Comparison of the distributions of angular values in held-out test set and in generated samples. Top row shows dihedral angles (torsional angles involving 4 atoms), and bottom row shows bond angles (involving 3 atoms). KL divergence is calculated between $D_{KL}(\text{sampled} || \text{test})$. Figure \ref{['fig:train_vs_generated_cdf']} shows the cumulative distribution function (CDF) corresponding to these histograms.
• Figure 3: Ramachandran plots comparing the $(\phi, \psi)$ dihedral angles for test set (\ref{['fig:ramachandran-test']}) and generated protein backbones (\ref{['fig:ramachandran-generated']}). Each major region of this plot indicates a different secondary structure element, as indicated in panel \ref{['fig:ramachandran-test']}. All three main structural elements are recapitulated in our generated backbones, along with some less common angle combinations. Lines are artifacts of null values, and appear shifted due to zero centering/uncentering.
• Figure 4: 2D histograms describing co-occurrence of secondary structures in test backbones (\ref{['fig:ss-cooccur-test']}) and generated backbones (\ref{['fig:ss-cooccur-sampled']}). Axes indicate the number of secondary structure present in a chain; color indicates the frequency of a specific combination of secondary structure elements. Our generated structures mirror real structures with multiple $\alpha$ helices, multiple $\beta$ sheets, and a mixture of both. See Figure \ref{['fig:ss-replicates']} for additional generation replicates.
• Figure 5: Of our 780 generated backbones, ranging in length from 50-128 residues, 177 are designable $(\mathrm{scTM} \geq 0.5)$ using ProteinMPNN and OmegaFold. Shorter structures of 70 amino acids or fewer tend to have higher scTM scores than longer structures (\ref{['fig:sctm-scores']}). Generated backbones that are more similar to training examples (greater maximum training TM score) tend to have better designability (\ref{['fig:sctm-vs-training-tm']}). The three structures indicated by arrows are illustrated in Figure \ref{['fig:addtl-generated-examples']}.