MACE Foundation Models for Lattice Dynamics: A Benchmark Study on Double Halide Perovskites

TL;DR

This study benchmarks four MACE foundation models against a DFT database of ~$2000$ cubic halide double perovskites to assess their ability to predict dynamic stability and vibrational anharmonicity. It integrates harmonic phonon analysis (via finite-displacement force constants) and finite-temperature AIMD-based anharmonicity quantified by $\sigma^{(2)}$, along with configurational-space comparisons using SOAP-REMatch and KPCA. The omat-0-medium model consistently offers the best alignment with DFT for both 0 K harmonic phonons and 300 K dynamics, with prediction accuracy correlating to the materials' anharmonicity and halide mass; Hessian-related errors dominate the discrepancies. The work demonstrates the viability of MACE foundation models for fast pre-screening of dynamic stability while highlighting the need for more diverse training data (including Hessians) to improve robustness, and it introduces the anharmonicity score as a physically informative diagnostic for model performance.

Abstract

Recent developments in materials informatics and artificial intelligence has led to the emergence of foundational energy models for material chemistry, as represented by the suite of MACE-based foundation models, bringing a significant breakthrough in universal potentials for inorganic solids. As to all method developments in computational materials science, performance benchmarking against existing high-level data with focusing on specific applications, is critically needed to understand the limitations in the models, thus facilitating the ongoing improvements in the model development process, and occasionally, leading to significant conceptual leaps in materials theory. Here, using our own published DFT (Density Functional Theory) database of room-temperature dynamic stability and vibrational anharmonicity for $\sim2000$ cubic halide double perovskites, we benchmarked the performances of four different variants of the MACE foundation models for screening the dynamic stabilities of inorganic solids. Our analysis shows that, as anticipated, the model accuracy improves with more training data. The dynamic stabilities of weakly anharmonic materials (as predicted by DFT) are more accurately reproduced by the foundation model, than those highly anharmonic and dynamically unstable ones. The predominant source of error in predicting the dynamic stability arises predominantly from the amplification of errors in atomic forces when predicting the harmonic phonon properties through the computation of the Hessian matrix, less so is the contribution from possible differences in the range of the configurational spaces that are sampled by DFT and the foundation model in molecular dynamics. We hope that our present findings will stimulate future works towards more physics-inspired approaches in assessing the accuracy of foundation models for atomistic modelling.

Figures (16)

• Figure 1: Accuracies of the MACE foundation models in predicting the harmonic phonon properties for HDPs. (a) Correlations between the RMSEs in predicting the phonon eigenfrequencies and group velocities with respect to the DFT results for chloride HDPs using the omat-0-medium foundation model. Each data point is colour-coded according to its anharmonicity score $\sigma^{(2)}$ computed from DFTyang2022mapping. Two extrema with the best (lower left) and worst (upper right) prediction accuracies are highlighted, the corresponding phonon dispersion relationships for which are shown in \ref{['fig:phonon_dispersion_compare_omat']}. (b) and (c) show the box plots that present the ranges of RMSEs in predicting the phonon eigenfrequencies and group velocities using all four foundation models for each groups of HDPs as categorised by the halide anion. The orange line indicates the medium RMSE. The box limits represent the 1st and 3rd quartiles. The whiskers show the range of the RMSEs within 1.5$\times$ the interquartile range of the box limits, and the outliers are denoted by light cyan plus symbols.
• Figure 2: Table of confusion matrices showing how well the MACE models (across the columns) reproduce the vibrational anharmoncity of DHPs computed from DFT (across the rows), which is represented as the number of DHPs that fall into each category. Top (bottom) row presents the anharmonicity scores evaluated using force constant matrix computed from DFT (corresponding MACE models shown on the top row).
• Figure 3: Confusion matrices showing the reproducibility of the DFT-computed vibrational anharmonicity for DHPs using the MACE model. Here, the omat-0-medium model is used for both computing the harmonic force constants, as well as the molecular dynamics samplings.
• Figure 5: KPCA maps for the five fluoride HDPs listed in \ref{['tab:kpca_compounds']} that compare the configuration spaces sampled by AIMDyang2022mapping and MACE-MD with the omat-0-medium model. Each point on the map corresponds to a configuration in the MD trajectory.
• Figure S1: Landscape of room-temperature vibrational anharmonicities as measured by $\sigma^{(2)}$ as a function of formation energies for HDPs, in which data results for HDPs with different halogen anions are separately colour-coded. (a) $\sigma^{(2)}$ plotted as a function of the formation energies $\Delta E_f$. (b) $\sigma^{(2)}$ plotted as a function of anharmonicity-weighted-averaged phonon frequency for each HDP, defined as $\langle\omega\rangle_\sigma=\sum_{\bm{q},n}\omega(\bm{q},n)\sigma^{(2)}(\bm{q},n)/\sum_{\bm{q},n}\sigma^{(2)}(\bm{q},n)$, in which the phonon-mode-resolved anharmonicity score $\sigma^{(2)}(\bm{q},n)$ was computed using the same definition as \ref{['eq:anharmonic_score']} except all the atomic forces are projected onto individual phonon eigenvectors $\bm{u}(\bm{q},n)$ [Reproduced from Yang et al.yang2022mapping].