SeedFold: Scaling Biomolecular Structure Prediction

TL;DR

SeedFold tackles the scalability challenge in biomolecular structure prediction by jointly scaling width, introducing linear triangular attention to reduce cubic complexity, and augmenting data with a large distillation corpus. The approach reveals width (hidden dimension of the pair representation) as the primary bottleneck and demonstrates substantial gains when expanding capacity, while linear attention offers a practical, memory-efficient alternative with competitive performance. On FoldBench, SeedFold achieves state-of-the-art or highly competitive results across multiple protein-related tasks, with the vanilla and linear attention variants showing complementary strengths across monomer and interface predictions. These results establish SeedFold as a scalable, high-accuracy folding model with potential impacts on protein foundation modeling and downstream applications requiring robust 3D structure prediction.

Abstract

Highly accurate biomolecular structure prediction is a key component of developing biomolecular foundation models, and one of the most critical aspects of building foundation models is identifying the recipes for scaling the model. In this work, we present SeedFold, a folding model that successfully scales up the model capacity. Our contributions are threefold: first, we identify an effective width-scaling strategy for the Pairformer to increase representation capacity; second, we introduce a novel linear triangular attention that reduces computational complexity to enable efficient scaling; finally, we construct a large-scale distillation dataset to substantially enlarge the training set. Experiments on FoldBench show that SeedFold outperforms AlphaFold3 on most protein-related tasks.

Figures (12)

• Figure 1: Overview of SeedFold. (i) We scale folding models from three perspectives. Model: scaling the Pairformer width to increase model capacity; Architecture: linear triangular attention reduces computational complexity while maintaining prediction quality; Data: large-scale distillation expands training data to $26.5$M samples. (ii) SeedFold denotes a 512-width model equipped with vanilla triangular attention, while SeedFold-Linear refers to a 384-width model with linear attention. (iii) SeedFold achieves state-of-the-art performance on FoldBench, surpassing AlphaFold3 and other open-source models on multiple tasks.
• Figure 2: Comparison of different scaling strategies. (a) Conceptual illustration of the scaling strategy.(b) Performance comparison where the left plot shows complex RMSD (lower is better) for global structural accuracy, and the right plot shows intra-protein lDDT (higher is better) for local structural quality. Width scaling consistently outperforms depth scaling. The transition from Small ($128$-width) to Medium ($256$-width) yields the largest gains in both global RMSD and local lDDT, while further scaling to Large ($512$-width) shows diminishing returns. Deeper trunk and deeper structure module provide marginal improvements compared to width scaling.
• Figure 3: The LinearTriangularAttention module. (a) The architecture of the linear attention module. (b) The peak memory usage (MB) and time cost (ms) of different attention modules. Two linear attention mechanisms have similar peak memory usages, which overlap in the figure.
• Figure 4: Cumulative distribution of interface prediction success rates across different modalities. We compare SeedFold and SeedFold-Linear against Boltz-1 wohlwend2025boltz and Protenix-0.5 bytedance2025protenix. Note that AlphaFold 3's detailed distribution metrics are not available due to license restrictions; their performance could be found in FoldBench xu2025benchmarking.
• Figure 5: The validation scores across various metrics for different types of triangular attention are presented as follows. Linear triangular attention achieves on-par results on most metrics; moreover, in RNA/DNA-related tasks, GatedLinearTriAtt outperforms AdditiveLinearTriAtt and the standard attention mechanism.