Cross-functional transferability in universal machine learning interatomic potentials

TL;DR

The paper tackles cross-functional transferability of universal MLIPs across DFT functionals (GGA/GGA+U and r^2SCAN) and identifies large energy-scale shifts as a key obstacle. It introduces a CHGNet-based transfer-learning approach that uses energy referencing via atomic reference energies (AtomRef) to align labels between functionals. The study finds that refitting AtomRef to the target functional dramatically improves inter-functional correlation and model accuracy, enabling data-efficient transfer even with sub-million high-fidelity data, and shows TL outperforms training from scratch across energy, forces, and stability predictions. It also highlights the need for multi-fidelity benchmarking when advancing uMLIPs to higher-accuracy functionals, outlining practical strategies for integrating diverse datasets to build next-generation interatomic potentials.

Abstract

The rapid development of universal machine learning interatomic potentials (uMLIPs) has demonstrated the possibility for generalizable learning of the universal potential energy surface. In principle, the accuracy of uMLIPs can be further improved by bridging the model from lower-fidelity datasets to high-fidelity ones. In this work, we analyze the challenge of this transfer learning problem within the CHGNet framework. We show that significant energy scale shifts and poor correlations between GGA and r$^2$SCAN pose challenges to cross-functional data transferability in uMLIPs. By benchmarking different transfer learning approaches on the MP-r$^2$SCAN dataset of 0.24 million structures, we demonstrate the importance of elemental energy referencing in the transfer learning of uMLIPs. By comparing the scaling law with and without the pre-training on a low-fidelity dataset, we show that significant data efficiency can still be achieved through transfer learning, even with a target dataset of sub-million structures. We highlight the importance of proper transfer learning and multi-fidelity learning in creating next-generation uMLIPs on high-fidelity data.

Figures (5)

• Figure 1: Statistical analysis of the energy data.a Element distribution of the MP-r$^2$SCAN dataset of 238,247 structures. The color indicates the total number of occurrences of an element in the MP-r$^2$SCAN dataset with a lower cutoff of 1000. b Total Energy of materials computed from GGA/GGA+U vs. r$^2$SCAN functionals. Each point represents a material with a materials ID that has r$^2$SCAN calculations in Materials Project, with the $x$-axis showing the total energy after r$^2$SCAN structure relaxation and the $y$-axis showing the total energy after GGA/GGA+U structure relaxation. The marginal histograms on the top and right illustrate the distributions of total energies for the same collection of materials, as calculated by r$^2$SCAN and GGA/GGA$+U$, respectively. c--d Feature importance in the formation energy differences between GGA/GGA$+U$ mixing and r$^2$SCAN. Each element is treated as a feature, with its importance indicated by colors on the periodic table. Higher values correspond to greater importance and therefore larger energy difference between GGA/GGA$+U$ and r$^2$SCAN. Panel c presents the feature importance when anion and compatibility corrections are included in the mixed GGA/GGA$+U$ data, and panel d presents the feature importance without these adjustments. Compositional corrections are applied primarily to pnictogens, chalcogens, and halogens.
• Figure 2: Illustration of AtomRef and correlation improvement through scaled energies.a Schematic representation of the role and application of AtomRef in calculating total energies. The energy contribution from AtomRef is obtained by taking the dot product of the composition row vector (with LiCoO$_2$ used here as an example) and the AtomRef vector. b The correlation between the scaled energies of GGA/GGA$+U$ and r$^2$SCAN (total energies with the respective AtomRefs subtracted). The marginal histograms on the top and right illustrate the distributions of r$^2$SCAN and GGA/GGA$+U$ scaled energies, respectively, for the same collection of materials.
• Figure 3: Comparison of the model's training performance with and without AtomRef refitting.a Gradient values recorded every 1/10 of an epoch for various model layers during the first transfer learning epoch, comparing models with and without AtomRef refitting. The layers include "AtomEmb" (atom embedding), "BondEmb" (bond embedding), "AngleEmb" (angle embedding), "AtomConv0_W0" and "AtomConv3_W3" (weights of the two-body atom convolution layers), "BondConv0_W0" and "BondConv2_W3" (weights of the two-body bond convolution layers), and "MLP_Layer0" (weights of the first layer in the multi-layer perceptron). b Energy training history for Method 3, showing the lowest energy MAE of 18.37 meV/atom at the last epoch. c Energy training history for Method 4, showing the lowest energy MAE of 11.82 meV/atom at the last epoch.
• Figure 4: Decomposition energy prediction workflow. The left plot shows a schematic of a convex hull energy diagram constructed using r$^2$SCAN DFT-calculated data, providing decomposition energy values based on competing phases identified in the DFT phase diagram (e.g., for a$_2$, the competing phases are a$_1$ and a$_3$; for a$_4$, they are a$_3$ and a$_6$). The right plot schematically shows the convex hull constructed by CHGNet-relaxed energies. The decomposition energy and model-identified competing phases differ from DFT.
• Figure 5: Scaling law on r$^2$SCAN data. a Energy MAE and b Force MAE on the MP-r$^2$SCAN validation set using either Method 4, TL with r$^2$SCAN AtomRef (Transfer, blue) or Method 1, training from scratch (Scratch, orange) methods. Zero training points in Transfer refers to the performance of the GGA/GGA$+U$ pre-trained CHGNet with r$^2$SCAN AtomRef. Linear fits are applied for $x>1000$ to demonstrate the neural scaling law, and the coefficients of determination (R$^{2}$) are shown in the figures.