Evaluation of Foundational Machine Learned Interatomic Potentials for Migration Barrier Predictions

TL;DR

This work benchmarks five foundational MLIPs—MACE-MP-0, Orb-v3, SevenNet, CHGNet, and M3GNet—integrated with NEB to predict ionic migration barriers $E_m$ against DFT-NEB across diverse battery-relevant chemistries. It provides a systematic assessment of barrier accuracy, barrier-range performance, classification of good versus bad ionic conductors, and local-geometry predictions through a novel geometric similarity metric, revealing that barrier accuracy and geometry predictions are largely decoupled. Key findings include Orb-v3 achieving the best non-outlier MAE of $0.198$ eV and Orb-v3/SevenNet attaining >$82 ext{%}$ accuracy for classifying conductors, along with a majority of MLIP-NEB relaxations improving the MEP relative to LI in over 66% of cases. The results offer practical guidance for using foundational MLIPs to accelerate discovery of ionic conductors while highlighting limitations for high-$E_m$ systems and the lack of correlation between barrier prediction and local geometry accuracy.$ E_m $ is the central observable, and $E_m$-related predictions enable faster screening and targeted refinement in battery materials research.

Abstract

Fast, and accurate prediction of ionic migration barriers ($E_m$) is crucial for designing next-generation battery materials that combine high energy density with facile ion transport. Given the computational costs associated with estimating $E_m$ using conventional density functional theory (DFT) based nudged elastic band (NEB) calculations, we benchmark the accuracy in $E_m$ and geometry predictions of five foundational machine learned interatomic potentials (MLIPs), which can potentially accelerate predictions of ionic transport. Specifically, we assess the accuracy of MACE-MP-0, Orb-v3, SevenNet, CHGNet, and M3GNet models, coupled with the NEB framework, against DFT-NEB-calculated $E_m$ across a diverse set of battery-relevant chemistries and structures. Notably, MACE-MP-0 and Orb-v3 exhibit the lowest mean absolute errors in $E_m$ predictions across the entire dataset and over data points that are not outliers, respectively. Importantly, Orb-v3 and SevenNet classify `good' versus `bad' ionic conductors with an accuracy of $>$82\%, based on a threshold $E_m$ of 500~meV, indicating their utility in high-throughput screening approaches. Notably, intermediate images generated by MACE-MP-0 and SevenNet provide better initial guesses relative to conventional interpolation techniques in $>$71\% of structures, offering a practical route to accelerate subsequent DFT-NEB relaxations. Finally, we observe that accurate $E_m$ predictions by MLIPs are not correlated with accurate (local) geometry predictions. Our work establishes the use-cases, accuracies, and limitations of foundational MLIPs in estimating $E_m$ and should serve as a base for accelerating the discovery of novel ionic conductors for batteries and beyond.

Figures (6)

• Figure 1: Overview of the methodology, indicating the use of two subsets of the $E_m$ dataset that were created for examining geometry predictions, geometry-barrier correlations, and barrier predictions.
• Figure 2: Parity plot of migration barrier predicted by various MLIPs against DFT-NEB, with the dotted black line indicating the parity line. Inset shows the parity plot for a smaller range of DFT/predicted values (0-2 eV).
• Figure 3: Barrier prediction performance of various MLIPs across different DFT-calculated $E_m$ ranges. The dotted line and kink denote a change in the models, which are, from top to bottom: MACE-MP-0, SevenNet, Orb-v3, CHGNet, and M3GNet. Each bin contains an equal number of data points with the width corresponding to the range of DFT-calculated $E_m$ within the bin. The height of each bin (as indicated by the numerical annotation on each bin) within each model represents the percentage of data points whose $E_m$ values are predicted within an absolute error of 0.1 eV.
• Figure 4: Confusion matrices for barrier prediction across different models. Each matrix corresponds to a specific model and is structured such that the upper-left, upper-right, lower-left, and lower-right cells represent the counts of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN), respectively. A prediction is considered a TP (TN) if both the DFT-computed and model-predicted $E_m$ are less than (greater than or equal to) 500 meV.
• Figure 5: Performance of MLIPs on local geometry prediction. Each entry in the heatmap represents a performance fraction for a given MLIP with the last column corresponding to IDPP. The top (bottom) row shows the fraction of structures classified as 'good' ('bad') to the total number of structures. The heatmap color bar varies from red (high fractions) to blue (low fractions).