Triangle Multiplication Is All You Need For Biomolecular Structure Representations

TL;DR

AlphaFold3–style backbones remain computationally intensive due to triangle attention, limiting large-scale use. Pairmixer replaces sequence updates and triangle attention with an attention-free backbone that relies on triangle multiplication and FFNs, preserving higher-order geometric reasoning. It achieves up to 4x faster long-sequence inference and a 34% reduction in training GPU-days while matching state-of-the-art benchmarks across folding, docking, and diverse biomolecular tasks, including binder design. This efficiency enables scalable, high-throughput structure prediction and design workflows, broadening the practical impact of computational structural biology; code is available at the provided repository.

Abstract

AlphaFold has transformed protein structure prediction, but emerging applications such as virtual ligand screening, proteome-wide folding, and de novo binder design demand predictions at a massive scale, where runtime and memory costs become prohibitive. A major bottleneck lies in the Pairformer backbone of AlphaFold3-style models, which relies on computationally expensive triangular primitives-especially triangle attention-for pairwise reasoning. We introduce Pairmixer, a streamlined alternative that eliminates triangle attention while preserving higher-order geometric reasoning capabilities that are critical for structure prediction. Pairmixer substantially improves computational efficiency, matching state-of-the-art structure predictors across folding and docking benchmarks, delivering up to 4x faster inference on long sequences while reducing training cost by 34%. Its efficiency alleviates the computational burden of downstream applications such as modeling large protein complexes, high-throughput ligand and binder screening, and hallucination-based design. Within BoltzDesign, for example, Pairmixer delivers over 2x faster sampling and scales to sequences ~30% longer than the memory limits of Pairformer. Code is available at https://github.com/genesistherapeutics/pairmixer.

Figures (16)

• Figure 1: Pairmixer is an efficient architecture for biomolecular structure prediction. Across multiple model sizes, Pairmixer matches the performance of the leading Pairformer architecture while delivering greater training efficiency.
• Figure 2: Overview of Biomolecular Structure Prediction. Given a list of sequences, our model predicts the 3D folded structure of all sequences within a single complex. Input sequences are first embedded into a single representation for each residue and a pair representation to capture the relationship between pairs of residues. The MSA Module and Backbone (e.g., Pairformer) extracts deep pairwise features capturing inter-residue interactions, which are then passed to the diffusion module to generate the 3D structure. (Additional inputs such as MSAs, conformers, and templates are omitted for clarity.)
• Figure 3: Schematic comparison of the Pairformer and Pairmixer backbones.Pairmixer simplifies the Pairformer architecture by removing redundancies. This results in faster training and inference, expanding the scale of downstream applications.
• Figure 4: Performance curves on RCSB test set across model sizes. We compare three backbone architectures across three model sizes over training. Pairmixer matches or surpasses the Pairformer baseline while training more efficiently.
• Figure 5: Inference speed analysis. We measure runtime across architectures and input sizes. While the Transformer is the fastest overall, Pairmixer achieves substantially lower inference times than Pairformer, particularly on longer sequences.