Platonic representation of foundation machine learning interatomic potentials

TL;DR

The paper proves that foundation MLIPs trained on overlapping chemical spaces develop a shared latent geometry despite architectural differences. It introduces the Platonic representation, projecting atomic embeddings into a common space using $K$ anchors via $\mathbf{z}_i = [\cos(\mathbf{e}_i,\mathbf{a}_1), \dots, \cos(\mathbf{e}_i,\mathbf{a}_K)]^\top$, with anchor sets generated by DIRECT sampling. Across seven diverse MLIPs, embeddings align into a coherent chemical geometry, enabling cross-model optimal transport and algebraic embedding arithmetic, including zero-shot model stitching, while also exposing representational biases and potential diagnostic signals for symmetry breaking. The framework offers a practical pathway toward interoperable, interpretable foundation potentials for materials science, and highlights the value of considering representational compatibility alongside predictive performance in model design.

Abstract

Foundation machine learning interatomic potentials (MLIPs) are trained on overlapping chemical spaces, yet their latent representations remain model-specific. Here, we show that independently developed MLIPs exhibit statistically consistent geometric organisation of atomic environments, which we term the Platonic representation. By projecting embeddings relative to a set of atomic anchors, we unify the latent spaces of seven MLIPs (spanning equivariant, non-equivariant, conservative, and non-conservative architectures) into a common metric space that preserves chemical periodicity and structural invariants. This unified framework enables direct cross-model optimal transport, interpretable embedding arithmetic, and the detection of representational biases. Furthermore, we demonstrate that geometric distortions in this space can indicate physical prediction failures, including symmetry breaking and incorrect phonon dispersions. Our results show that the latent spaces of diverse MLIPs present consistent statistical geometry shaped by shared physical and chemical constraints, suggesting that the Platonic representation offers a practical route toward interoperable, comparable, and interpretable foundation models for materials science.

Figures (5)

• Figure 1: Model-specific embeddings are incompatible before transformation. 2D-PCA projections of atomic embeddings from seven foundation MLIPs reveal distinct variance directions and element clustering patterns. Although all models are trained to predict the same physical quantities for overlapping material sets, they learn embeddings in incompatible coordinate systems.
• Figure 2: Construction of the unified coordinate system. (a) Schematic of the anchor-based transformation. Green, red, and purple points represent a set of three anchor vectors ($\mathbf{a}_k$), while the yellow point indicates a sample vector $\mathbf{e}_i \in \mathbb{R}^d$. The transformation projects $\mathbf{e}_i$ into the anchor-defined space via cosine similarity. (b) Distribution of 100 DIRECT-sampled anchors (blue dots) overlaid on the PCA projection of the original embedding manifold (orange background).
• Figure 3: Variation of transformed representations with anchor set size and sampling strategy. (a) Transformed representations as a function of anchor set size ($K = 3$ to $400$). (b) 2D PCA projections of converged representations using 100 randomly sampled anchors. (c) Projections using 100 DIRECT-sampled anchors. Despite architectural diversity, all models transformed with DIRECT sampling show substantial alignment. Non-equivariant models (Orb-v3) exhibit systematic skewness. The Dummy model (untrained, random weights) displays no chemical structure, confirming that alignment reflects learned physical knowledge. Colourmap follows the labelling in Figure 1.
• Figure 4: Quantifying model similarity and chemical preservation. (a) mKNN scores (local fidelity), (b) Procrustes scores (global alignment), (c) Normalized Optimal Transport (OT) cost, and (d) the composite SuperScore. (e, f) Element-level embeddings projected into the unified space ($K$=100 anchors) reveal consistent periodic clustering across all seven models (Fig. S7), here results with MACE-large (e) and conservative Orb model (f) are presented. Equivariant architectures (MACE, SevenNet) produce compact clusters, whereas non-equivariant models (Orb-v3) exhibit skewness, suggesting equivariance constraints sharpen chemical organization.
• Figure 5: Diagnostic applications via Platonic embeddings. (a) Initial PCA projection of Cu (stars) and Au (crosses) embeddings across MACE models. (b) Naive fine-tuning (ft1) on additional Cu data causes embeddings for unseen Au atoms to collapse toward the origin (purple trajectories). (c) Multi-head fine-tuning (ft2) keeps unseen Au embeddings stable near their pre-trained positions (teal trajectories). (d) Naive embedding collapse results in erroneous, repulsive potentials for Au–Au dimers (green line). (e) This strategy preserves correct Au–Au physics while selectively adapting Cu–Cu interactions to the target (red line). (f) The equivariant fraction of embeddings from different models based on two-nearest-neighbour analysis. (g) Rotational sensitivity test on BaCeO3: Equivariant models preserve embeddings under rotation; Orb-v3 models do not.