Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Inter-Atomic Potentials without Explicit Equivariance

Ahmed A. Elhag, Arun Raja, Alex Morehead, Samuel M. Blau, Garrett M. Morris, Michael M. Bronstein

TL;DR

This work tackles the challenge of learning accurate interatomic potentials without explicit $SO(3)$-equivariant architectures. It introduces TransIP, a Transformer-based framework that learns latent $SO(3)$-equivariance through a transformation network and a contrastive latent-loss objective, trained on the diverse OMol25 dataset. Empirical results show TransIP achieves 40–60% performance gains over data-augmentation baselines, scales effectively with data and model size, and can rival traditional equivariant models while preserving computational efficiency. The approach offers a data-driven, scalable alternative to architectural equivariance for MLIPs, with meaningful implications for large-scale molecular simulations.

Abstract

Accurate and scalable machine-learned inter-atomic potentials (MLIPs) are essential for molecular simulations ranging from drug discovery to new material design. Current state-of-the-art models enforce roto-translational symmetries through equivariant neural network architectures, a hard-wired inductive bias that can often lead to reduced flexibility, computational efficiency, and scalability. In this work, we introduce TransIP: Transformer-based Inter-Atomic Potentials, a novel training paradigm for interatomic potentials achieving symmetry compliance without explicit architectural constraints. Our approach guides a generic non-equivariant Transformer-based model to learn SO(3)-equivariance by optimizing its representations in the embedding space. Trained on the recent Open Molecules (OMol25) collection, a large and diverse molecular dataset built specifically for MLIPs and covering different types of molecules (including small organics, biomolecular fragments, and electrolyte-like species), TransIP effectively learns symmetry in its latent space, providing low equivariance error. Further, compared to a data augmentation baseline, TransIP achieves 40% to 60% improvement in performance across varying OMol25 dataset sizes. More broadly, our work shows that learned equivariance can be a powerful and efficient alternative to augmentation-based MLIP models.

Learning Inter-Atomic Potentials without Explicit Equivariance

TL;DR

This work tackles the challenge of learning accurate interatomic potentials without explicit -equivariant architectures. It introduces TransIP, a Transformer-based framework that learns latent -equivariance through a transformation network and a contrastive latent-loss objective, trained on the diverse OMol25 dataset. Empirical results show TransIP achieves 40–60% performance gains over data-augmentation baselines, scales effectively with data and model size, and can rival traditional equivariant models while preserving computational efficiency. The approach offers a data-driven, scalable alternative to architectural equivariance for MLIPs, with meaningful implications for large-scale molecular simulations.

Abstract

Accurate and scalable machine-learned inter-atomic potentials (MLIPs) are essential for molecular simulations ranging from drug discovery to new material design. Current state-of-the-art models enforce roto-translational symmetries through equivariant neural network architectures, a hard-wired inductive bias that can often lead to reduced flexibility, computational efficiency, and scalability. In this work, we introduce TransIP: Transformer-based Inter-Atomic Potentials, a novel training paradigm for interatomic potentials achieving symmetry compliance without explicit architectural constraints. Our approach guides a generic non-equivariant Transformer-based model to learn SO(3)-equivariance by optimizing its representations in the embedding space. Trained on the recent Open Molecules (OMol25) collection, a large and diverse molecular dataset built specifically for MLIPs and covering different types of molecules (including small organics, biomolecular fragments, and electrolyte-like species), TransIP effectively learns symmetry in its latent space, providing low equivariance error. Further, compared to a data augmentation baseline, TransIP achieves 40% to 60% improvement in performance across varying OMol25 dataset sizes. More broadly, our work shows that learned equivariance can be a powerful and efficient alternative to augmentation-based MLIP models.

Paper Structure

This paper contains 24 sections, 17 equations, 10 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Symmetry in Embedding Space
  3. Problem Formulation
  4. Implicit Equivariance in Embedding Space
  5. Learning Inter-Atomic Potentials without Explicit Equivariance
  6. TransIP: Transformer-based Interatomic Potentials
  7. Learned Latent Equivariance
  8. Training Objective
  9. Related Work
  10. Experimental Setup
  11. Results and Discussion
  12. Scaling data size
  13. Learned latent equivariance
  14. Architectural equivariance versus learned equivariance
  15. Conclusion
  16. ...and 9 more sections

Figures (10)

  • Figure 1: TransIP: Transformer-based Interatomic Potentials.
  • Figure 2: Val-Comp performance across different dataset sizes (1M / 2M / 4M): The top row presents force metrics, while the bottom row reports energy metrics.
  • Figure 3: Latent equivariance (embedding) error versus validation performance. The top row reports force metrics, while the bottom row presents energy metrics.
  • Figure 4: Validation total inference trade-off (atoms/s versus performance). The top row presents force metrics, while the bottom row represents energy metrics.
  • Figure 5: Metal Complexes scaling across training dataset sizes (1M / 2M / 4M). The top row presents force metrics, while the bottom row displays energy metrics.
  • ...and 5 more figures