Physio-DPO: Aligning Large Language Models with the Protein Energy Landscape to Eliminate Structural Hallucinations

TL;DR

Physio-DPO tackles structural hallucinations in large protein language models by introducing a physics-informed alignment that couples thermodynamic stability to preference optimization. It extends Direct Preference Optimization with a magnitude-aware energy weighting, using an energy gap $\delta_E$ and a sigmoid-based weight $\Psi(\delta_E)$ to scale updates, and employs hard-negative mining to expose subtle biophysical failures. The authors construct the PhysioPref-1M benchmark and provide theoretical analysis showing gradient variance reduction and alignment to a Boltzmann-like energy distribution, alongside extensive experiments demonstrating significant improvements in sc-RMSD (1.28 Å) and foldability (92.8%) with qualitative recovery of hydrogen bonds and hydrophobic core packing. This physics-grounded alignment enables scalable, biophysically valid protein generation while preserving linguistic diversity, suggesting a practical path to more reliable protein design with PLMs.

Abstract

Large Protein Language Models have shown strong potential for generative protein design, yet they frequently produce structural hallucinations, generating sequences with high linguistic likelihood that fold into thermodynamically unstable conformations. Existing alignment approaches such as Direct Preference Optimization are limited in this setting, as they model preferences as binary labels and ignore the continuous structure of the physical energy landscape. We propose Physio-DPO, a physics informed alignment framework that grounds protein language models in thermodynamic stability. Physio-DPO introduces a magnitude aware objective that scales optimization updates according to the energy gap between native structures and physics perturbed hard negatives. Experiments show that Physio-DPO consistently outperforms strong baselines including SFT, PPO, and standard DPO, reducing self consistency RMSD to 1.28 Å and increasing foldability to 92.8%. Qualitative analysis further demonstrates that Physio-DPO effectively mitigates structural hallucinations by recovering biophysical interactions such as hydrophobic core packing and hydrogen bond networks.

Figures (6)

• Figure 1: Ovewview of Physio-DPO framework. The physics stream folds a sampled pair $(y_w, y_l)$ and computes an energy gap $\delta_E$, which is mapped to a weight $\Psi(\delta_E)$. The language modeling stream computes the DPO log ratio using the policy $\pi_\theta$ and a frozen reference $\pi_{\mathrm{ref}}$. Physio-DPO reweights each DPO term by $\Psi(\delta_E)$, amplifying updates from pairs with large stability gaps and aligning the model with a continuous energy landscape.
• Figure 2: Construction pipeline for PhysioPref-1M. Step 1: diverse sequence generation from a reference language model ferruz2022protgpt2; Step 2: structure prediction via folding; Step 3: scoring and labeling based on pLDDT confidence fang2025alphafold and Rosetta energy scores alford2017rosetta, including the identification of hard negatives with high confidence but poor stability; Step 4: construction of preference pairs that maximize stability gaps; and Step 5: human-in-the-loop evaluation to verify alignment between labels and biophysical judgment.
• Figure 3: Training dynamics curves, (a) Physical Energy, (b) KL Divergence, (c) sc-RMSD.
• Figure 4: Energy vs. Confidence (pLDDT) plane.
• Figure 5: Comprehensive qualitative analysis of biophysical validity. We compare structures generated by the SFT Baseline (Red/Pink/Grey, Left) and Physio-DPO (Blue/Teal, Right). (a) Physio-DPO compacts disordered loops into stable helices. (b) Exposed hydrophobic residues (Orange) in SFT are buried into a tight core (Teal) by Physio-DPO. (c, e, f) Restoration of critical atomic interactions: hydrogen bond networks in $\beta$-sheets, precise disulfide bond geometry ($2.05\text{\AA}$), and electrostatic salt bridges. (d, g, i) Correction of severe geometrical violations, including non-physical "sawtooth" backbones, forbidden torsion kinks, and chain connectivity breaks. (h, j) Optimization of packing density by eliminating destabilizing internal voids and steric clashes (Red).