Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conditioning Protein Generation via Hopfield Pattern Multiplicity

Jeffrey D. Varner

Abstract

Protein sequence generation via stochastic attention produces plausible family members from small alignments without training, but treats all stored sequences equally and cannot direct generation toward a functional subset of interest. We show that a single scalar parameter, added as a bias to the sampler's attention logits, continuously shifts generation from the full family toward a user-specified subset, with no retraining and no change to the model architecture. A practitioner supplies a small set of sequences (for example, hits from a binding screen) and a multiplicity ratio that controls how strongly generation favors them. The method is agnostic to what the subset represents: binding, stability, specificity, or any other property. We find that the conditioning is exact at the level of the sampler's internal representation, but that the decoded sequence phenotype can fall short because the dimensionality reduction used to encode sequences does not always preserve the residue-level variation that defines the functional split. We term this discrepancy the calibration gap and show that it is predicted by a simple geometric measure of how well the encoding separates the functional subset from the rest of the family. Experiments on five Pfam families (Kunitz, SH3, WW, Homeobox, and Forkhead domains) confirm the monotonic relationship between separation and gap across a fourfold range of geometries. Applied to omega-conotoxin peptides targeting a calcium channel involved in pain signaling, curated seeding from 23 characterized binders produces over a thousand candidates that preserve the primary pharmacophore and all experimentally identified binding determinants. These results show that stochastic attention enables practitioners to expand a handful of experimentally characterized sequences into diverse candidate libraries without retraining a generative model.

Conditioning Protein Generation via Hopfield Pattern Multiplicity

Abstract

Protein sequence generation via stochastic attention produces plausible family members from small alignments without training, but treats all stored sequences equally and cannot direct generation toward a functional subset of interest. We show that a single scalar parameter, added as a bias to the sampler's attention logits, continuously shifts generation from the full family toward a user-specified subset, with no retraining and no change to the model architecture. A practitioner supplies a small set of sequences (for example, hits from a binding screen) and a multiplicity ratio that controls how strongly generation favors them. The method is agnostic to what the subset represents: binding, stability, specificity, or any other property. We find that the conditioning is exact at the level of the sampler's internal representation, but that the decoded sequence phenotype can fall short because the dimensionality reduction used to encode sequences does not always preserve the residue-level variation that defines the functional split. We term this discrepancy the calibration gap and show that it is predicted by a simple geometric measure of how well the encoding separates the functional subset from the rest of the family. Experiments on five Pfam families (Kunitz, SH3, WW, Homeobox, and Forkhead domains) confirm the monotonic relationship between separation and gap across a fourfold range of geometries. Applied to omega-conotoxin peptides targeting a calcium channel involved in pain signaling, curated seeding from 23 characterized binders produces over a thousand candidates that preserve the primary pharmacophore and all experimentally identified binding determinants. These results show that stochastic attention enables practitioners to expand a handful of experimentally characterized sequences into diverse candidate libraries without retraining a generative model.
Paper Structure (18 sections, 1 theorem, 10 equations, 8 figures, 14 tables)

This paper contains 18 sections, 1 theorem, 10 equations, 8 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Results
  3. Hard curation and the small-set amplifier.
  4. Multiplicity-weighted conditioning and the calibration gap.
  5. Cross-family validation and the separation index.
  6. Application to $\omega$-conotoxin Cav2.2 binders.
  7. AlphaFold2 multimer complex prediction.
  8. Structural and linguistic validation.
  9. Discussion
  10. Materials and Methods
  11. From Hopfield Energy to Multiplicity-Weighted Score Function
  12. Multiplicity Ratio and Phase Transition
  13. The Calibration Gap
  14. Author contributions.
  15. Competing interests.
  16. ...and 3 more sections

Key Result

Proposition 1

The score function of $p_{\mathbf{r}}(\bm{\xi}) \propto \exp(-\beta\,E_{\mathbf{r}}(\bm{\xi}))$ is $\nabla_{\bm{\xi}}\log p_{\mathbf{r}}(\bm{\xi}) = \mathbf{X}\,\mathop{\mathrm{softmax}}\nolimits(\beta\,\mathbf{X}^{\!\top}\bm{\xi} + \log\mathbf{r}) - \bm{\xi}$, where $\log\mathbf{r} = (\log r_1, \ld

Figures (8)

  • Figure 1: Diversity and composition fidelity improve with more input binders; phenotype fidelity is perfect at every sample size. Subsampling $K_{\mathrm{des}} \in \{3,5,8,10,15,20,25,32\}$ sequences from the Kunitz strong-binder set (3 replicates each; shaded bands show $\pm 1$ s.d.). P1 K/R fraction was 1.0 at every sample size (not shown). Left: Pairwise sequence diversity increases from 0.36 ($K_{\mathrm{des}}=3$) to 0.50 ($K_{\mathrm{des}}=32$). Right: KL divergence of amino acid composition decreases from 0.047 to 0.009 over the same range. Three input binders suffice for perfect phenotype transfer; more binders improve the compositional and diversity quality of the generated library.
  • Figure 2: Phase transition shifts rightward with increasing multiplicity ratio $\rho$. Attention entropy $H(\beta)$ normalized by $\log K$ (the uniform limit) as a function of inverse temperature $\beta$ (log scale) for $\rho \in \{1, 5, 20, 100, 1000\}$ on the Kunitz family ($K=99$, $K_{\mathrm{des}}=32$). At $\rho=1$ (standard SA), the entropy starts at $H/\log K = 1$ (uniform attention over all patterns); as $\rho$ increases, the multiplicity bias pre-concentrates the attention distribution, lowering the starting entropy toward $\log K_{\mathrm{des}} / \log K \approx 0.75$ (dotted line). Each curve shows a sigmoidal drop to near-zero entropy (retrieval phase); dots and dashed vertical lines mark the inflection point $\beta^{*}(\rho)$. As $\rho$ increases from 1 to 1000, $K_{\mathrm{eff}}$ decreases from 99 to 32.1 and $\beta^{*}$ shifts rightward from 4.4 to 9.3, consistent with fewer effective attractors requiring higher inverse temperature to produce an ordered phase.
  • Figure 3: Calibration trajectories reveal family-dependent gaps predicted by PCA-space separation.(A) Observed phenotype fraction $f_{\mathrm{obs}}$ versus effective designated fraction $f_{\mathrm{eff}}$ as the multiplicity ratio $\rho$ increases from 1 to 500 for six protein families. The dashed diagonal marks perfect calibration ($f_{\mathrm{obs}} = f_{\mathrm{eff}}$). Homeobox ($S = 0.42$) and SH3 ($S = 0.34$) domains track the diagonal closely; Kunitz ($S = 0.20$) and Forkhead ($S = 0.17$) curve below; WW domains ($S = 0.11$) show the largest departure; $\omega$-conotoxin ($S = 0.78$) shows a moderate gap despite high PCA separation. Double-headed arrows indicate the calibration gap at $\rho = 500$. (B) Fisher separation index $S$ versus calibration gap $\Delta$ at $\rho = 500$. A linear fit to the five Pfam families (filled markers; $\Delta \approx 0.72 - 1.8\,S$, $R^{2} = 0.80$) shows that the gap is predicted by the geometric separation of functional subsets in PCA space: families with $S > 0.3$ have gaps near zero, while families with $S < 0.2$ have gaps of $0.27$--$0.64$. The $\omega$-conotoxin point (open diamond) follows the monotonic trend.
  • Figure 4: Binding-loop amino acid frequencies and generation residuals for the $\omega$-conotoxin family (positions 9--17).Left column: Per-position amino acid frequency heatmaps for the input sequences (strong Cav2.2 binders, $n=23$, panel a; full O-superfamily, $n=74$, panel c). Right column: Residual heatmaps (generated $-$ input) showing the change in frequency at each position--residue pair after SA generation. Panel (b): strong-seeded residuals are near-zero throughout ($|\Delta f| < 0.15$), confirming that the Tyr pharmacophore at position 13, the conserved Cys framework (positions 14--16), and the variable-loop diversity (positions 9--12) are reproduced with high fidelity. Panel (d): full-seeded residuals are slightly larger, reflecting the broader sequence diversity of the input. Blue: enrichment; red: depletion; white: no change.
  • Figure 5: Sequence-level analysis of SA-generated Kunitz domain sequences conditioned on strong trypsin binders.(A) Alignment of the family consensus with the five highest-confidence SA strong-binder-conditioned sequences (ranked by ESMFold pLDDT). Positions are colored by conservation in the stored MSA: highly conserved ($>90\%$, red), conserved ($70$--$90\%$, orange), moderate ($50$--$70\%$, yellow), and variable ($<50\%$, blue). Cysteines forming the three disulfide bonds are highlighted in gold; the P1 position is highlighted in purple. Dots indicate matches to the consensus; letters indicate substitutions. All five sequences preserve every highly conserved position ($11/11$), all six cysteines ($6/6$), and carry K or R at P1, while introducing $19$--$26$ substitutions concentrated at variable sites. (B) Per-position Shannon entropy for the stored MSA (gray), SA full-family generation (blue), and SA strong-binder-conditioned generation (red). The full-family entropy correlation is very high ($r = 0.988$), confirming that SA recapitulates the position-specific conservation pattern of the natural family. The strong-conditioned entropy correlation is slightly lower ($r = 0.932$), with the largest deviation at the P1 position (column 25): conditioning collapses P1 diversity to K/R, reducing entropy at this site while preserving the entropy profile elsewhere.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Proposition 1: Score function
  • Remark 2: Phase transition: analytic baseline and multiplicity shift