LAMBench: A Benchmark for Large Atomistic Models

TL;DR

LAMBench introduces a cross-domain benchmarking framework to evaluate Large Atomistic Models (LAMs) on generalizability, adaptability, and applicability to approach the universal PES. It quantifies generalization with dimensionless, domain-aggregated metrics across force-field and domain-specific property tasks, and assesses practicality via efficiency and stability metrics. The study finds a sizable gap to universality, advocates for diverse, multi-domain and multi-fidelity training, conservativeness in energy predictions, and enhanced adaptability for downstream property tasks. As a dynamic, open platform with an interactive leaderboard, LAMBench aims to accelerate the development of robust, generalizable LAMs for broad scientific use.

Abstract

Large Atomistic Models (LAMs) have undergone remarkable progress recently, emerging as universal or fundamental representations of the potential energy surface defined by the first-principles calculations of atomistic systems. However, our understanding of the extent to which these models achieve true universality, as well as their comparative performance across different models, remains limited. This gap is largely due to the lack of comprehensive benchmarks capable of evaluating the effectiveness of LAMs as approximations to the universal potential energy surface. In this study, we introduce LAMBench, a benchmarking system designed to evaluate LAMs in terms of their generalizability, adaptability, and applicability. These attributes are crucial for deploying LAMs as ready-to-use tools across a diverse array of scientific discovery contexts. We benchmark ten state-of-the-art LAMs released prior to August 1, 2025, using LAMBench. Our findings reveal a significant gap between the current LAMs and the ideal universal potential energy surface. They also highlight the need for incorporating cross-domain training data, supporting multi-fidelity modeling, and ensuring the models' conservativeness and differentiability. As a dynamic and extensible platform, LAMBench is intended to continuously evolve, thereby facilitating the development of robust and generalizable LAMs capable of significantly advancing scientific research. The LAMBench code is open-sourced at https://github.com/deepmodeling/lambench, and an interactive leaderboard is available at https://www.aissquare.com/openlam?tab=Benchmark.

Figures (7)

• Figure 1: The schematic plot of the LAMBench benchmark.
• Figure 2: Dimensionless error metrics for generalizability tasks across different domains. (a) Dimensionless error metrics for force field tasks across three distinct domains. (b) Dimensionless error metrics for property-calculation tasks. To reassess DPA-3.1-3M with enhanced domain alignment, referred to as DPA-3.1-3M-DomainMatch, we employed task heads trained on SPICE2 for the Molecules domain, OMAT24 for the Inorganic Materials domain, and OC20M for the Catalysis domain.
• Figure 3: Distribution of inference time, normalized by the number of atoms, measured across 900 randomly selected configurations. Lower values indicate higher efficiency.
• Figure S-1: Structures used for the Conservativeness Benchmark. (a) H$_{2}$Al$_{32}$Cr$_{48}$Mn$_{16}$N$_{2}$O; (b) Cs$_{8}$N$_{2}$; (c) Gd$_{2}$Ni$_{2}$Si$_{4}$; (d) NdPd$_{3}$; (e) BaNi$_{2}$O$_{8}$V$_{2}$; (f) BaNiO$_{5}$Tm$_{2}$; (g) CH$_{3}$N$_{5}$S; (h) C$_{3}$H$_{5}$N$_{2}$; (i) C$_{4}$H$_{7}$NO.
• Figure S-2: The structures in unit cell for convergence test in efficiency benchmark. (a) H Cu Mg$_{11}$ O$_{12}$; (b) Na; (c) High-entropy alloy: Ag$_{2}$ Au Co Cr Cu Fe$_{2}$ Hf Ir Lu Mn Mo$_{2}$ Nb$_{2}$ Ni$_{2}$ Pd Pt$_{2}$ Rh Ru Sc$_{2}$ Ta$_{2}$ Ti$_{2}$ V W$_{3}$ Y Zn Zr; (d) H$_{2}$ Cl Cr$_{2}$ F O$_{10}$ Pb$_{4}$; (e) BN; (f) C Ni$_{36}$ O$_{2}$.