Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Benchmarking Universal Machine Learning Interatomic Potentials for Elastic Property Prediction

Pengfei Gao, Haidi Wang

TL;DR

The paper benchmarks four universal ML interatomic potentials (CHGNet, MACE, MatterSim, SevenNet) on elastic-property predictions across roughly 11k Materials Project structures, using DFT as reference. Elastic constants are extracted from stress–strain responses and converted to bulk, shear, Young's moduli, and Poisson's ratio via Voigt–Reuss–Hill averaging, enabling a rigorous comparison of accuracy and efficiency. SevenNet delivers the best overall accuracy, while MACE and MatterSim provide favorable accuracy–efficiency trade-offs; CHGNet generally underperforms and exhibits systematic biases. Targeted fine-tuning with strained configurations improves predictive performance for CHGNet, MatterSim, and SevenNet, demonstrating the value of non-equilibrium data in reducing biases and enhancing robustness for mechanical-property predictions, with MACE showing mixed gains. These results offer practical guidance for selecting uMLIPs in elastic-property applications and highlight directions for dataset augmentation and active-learning strategies to advance reliable, scalable materials design.

Abstract

Universal machine learning interatomic potentials have emerged as efficient tools for materials simulation, yet their reliability for elastic property prediction remains unclear. Here, we present a systematic benchmark of four uMLIPs -- MatterSim, MACE, SevenNet, and CHGNet -- against first-principles data for nearly 11\,000 elastically stable materials from the Materials Project database. The results show that SevenNet achieves the highest accuracy, MACE and MatterSim balance accuracy with efficiency, while CHGNet performs less effectively overall. To further improve predictive quality, we perform targeted fine-tuning on all four uMLIPs using strained configurations derived from 185 high-error materials. After fine-tuning, CHGNet exhibits the largest overall improvement, with an average mean absolute percentage error reduction of about 23\%, followed by MatterSim at around 21\% and SevenNet at 18\%, whereas MACE shows a performance degradation of roughly 14\%. This work provides quantitative guidance for model selection and data refinement, advancing uMLIPs toward reliable applications in mechanical property prediction.

Benchmarking Universal Machine Learning Interatomic Potentials for Elastic Property Prediction

TL;DR

The paper benchmarks four universal ML interatomic potentials (CHGNet, MACE, MatterSim, SevenNet) on elastic-property predictions across roughly 11k Materials Project structures, using DFT as reference. Elastic constants are extracted from stress–strain responses and converted to bulk, shear, Young's moduli, and Poisson's ratio via Voigt–Reuss–Hill averaging, enabling a rigorous comparison of accuracy and efficiency. SevenNet delivers the best overall accuracy, while MACE and MatterSim provide favorable accuracy–efficiency trade-offs; CHGNet generally underperforms and exhibits systematic biases. Targeted fine-tuning with strained configurations improves predictive performance for CHGNet, MatterSim, and SevenNet, demonstrating the value of non-equilibrium data in reducing biases and enhancing robustness for mechanical-property predictions, with MACE showing mixed gains. These results offer practical guidance for selecting uMLIPs in elastic-property applications and highlight directions for dataset augmentation and active-learning strategies to advance reliable, scalable materials design.

Abstract

Universal machine learning interatomic potentials have emerged as efficient tools for materials simulation, yet their reliability for elastic property prediction remains unclear. Here, we present a systematic benchmark of four uMLIPs -- MatterSim, MACE, SevenNet, and CHGNet -- against first-principles data for nearly 11\,000 elastically stable materials from the Materials Project database. The results show that SevenNet achieves the highest accuracy, MACE and MatterSim balance accuracy with efficiency, while CHGNet performs less effectively overall. To further improve predictive quality, we perform targeted fine-tuning on all four uMLIPs using strained configurations derived from 185 high-error materials. After fine-tuning, CHGNet exhibits the largest overall improvement, with an average mean absolute percentage error reduction of about 23\%, followed by MatterSim at around 21\% and SevenNet at 18\%, whereas MACE shows a performance degradation of roughly 14\%. This work provides quantitative guidance for model selection and data refinement, advancing uMLIPs toward reliable applications in mechanical property prediction.
Paper Structure (14 sections, 9 equations, 12 figures, 2 tables)

This paper contains 14 sections, 9 equations, 12 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset Construction and Analysis
  4. Brief Description of Evaluated uMLIPs
  5. Elastic Property Calculations
  6. Results and Analysis
  7. Model Performance Analysis
  8. Systematic Error Analysis
  9. Fundamental Limitations and the Impact of Fine-Tuning
  10. Discussion
  11. Implications for Materials Design Applications
  12. Future Directions
  13. Conclusions
  14. Supporting Information

Figures (12)

  • Figure 1: Crystal structure analysis of the dataset. (a) Periodic table heatmap indicating element occurrence. (b) Sunburst plot illustrating the distribution of crystal systems and space groups, with integers placed at the outer margins representing the corresponding space-group numbers. (c) Histogram of atom counts per unit cell.
  • Figure 2: Scatter plot of formation energy versus band gap, color-coded by the energy above hull. Marginal histograms illustrate the distribution of formation energies and band gaps. (b) Elastic property correlations. Scatter plot of bulk modulus versus shear modulus (log scale), color-coded by Poisson's ratio. Marginal histograms show the distributions of bulk and shear moduli.
  • Figure 3: Distributions of (a) bulk modulus, (b) shear modulus, (c) Young's modulus, and (d) Poisson's ratio, computed as Voigt--Reuss--Hill (VRH) averages from DFT and the four uMLIP models. Each violin plot shows the overall distribution, with blue and red dashed lines marking the median and mean, and short lines denoting the extrema.
  • Figure 4: (a)--(h) Scatter-plot comparison of four uMLIPs against DFT reference values for primary elastic properties—bulk modulus, shear modulus, Young's modulus, and Poisson's ratio, all computed as VRH averages. Each panel shows DFT values on the x-axis and model predictions on the y-axis, with the dashed line representing perfect agreement.
  • Figure 5: Elastic stability classification analysis comparing DFT and uMLIP models. (a) Stability classification rate showing the percentage of materials predicted as stable by each model. (b) Pairwise stability-agreement matrix, showing the percentage of materials assigned the same stability outcome across model pairs. Stability was determined using reported elastic stability flags or Born mechanical stability criteria.
  • ...and 7 more figures