A Graph Neural Network for the Era of Large Atomistic Models

TL;DR

This work introduces DPA3, a graph neural network built on line graph series (LiGS) to realize scalable, physically consistent large atomistic models (LAMs). By exploiting a multi-layer LiGS with dataset-encoding multitask training, DPA3 demonstrates formal scaling laws, enabling generalization improvements as model size, data, and compute increase, while maintaining energy conservation and symmetry invariances. The study validates DPA3 across diverse benchmarks (molecules, bulk materials, catalysts, 2D and battery materials) and shows that a large multitask model, DPA-3.1-3M, achieves state-of-the-art zero-shot force-field performance on 12 downstream tasks when trained on OpenLAM-v1. The work also discusses architectural choices (like SiLUT activation and residual updates), highlights the benefits of dataset encoding for scalability, and outlines paths for future improvements, including incorporating equivariant features and expanding training data to further enhance generalizability. Overall, DPA3 positions itself as a strong out-of-the-box PES model for the era of large atomistic models with robust transferability and adherence to fundamental physical laws.

Abstract

Foundation models, or large atomistic models (LAMs), aim to universally represent the ground-state potential energy surface (PES) of atomistic systems as defined by density functional theory (DFT). The scaling law is pivotal in the development of large models, suggesting that their generalizability in downstream tasks consistently improves with increased model size, expanded training datasets, and larger computational budgets. In this study, we present DPA3, a multi-layer graph neural network founded on line graph series (LiGS), designed explicitly for the era of LAMs. We demonstrate that the generalization error of the DPA3 model adheres to the scaling law. The scalability in the number of model parameters is attained by stacking additional layers within DPA3. Additionally, the model employs a dataset encoding mechanism that decouples the scaling of training data size from the model size within its multi-task training framework. When trained as problem-oriented potential energy models, the DPA3 model exhibits superior accuracy in the majority of benchmark cases, encompassing systems with diverse features, including molecules, bulk materials, surface and cluster catalysts, two-dimensional materials, and battery materials. When trained as a LAM on the OpenLAM-v1 dataset, the DPA-3.1-3M model exhibits lowest overall zero-shot generalization error across 12 downstream tasks spanning a diverse array of research domains. This performance suggests superior accuracy as an out-of-the-box potential model, requiring minimal fine-tuning data for downstream scientific applications.

Figures (9)

• Figure 1: Schematic plot of the DPA3 model architecture. (a) The line graph transform. (b) The line graph series (LiGS). (c) The model architecture of DPA3, a graph neural network on LiGS. (d) The update block of graph $G^{(1)}$. (e) The update block of graph $G^{(k)}$, $k>1$.
• Figure 2: Comparative performance of DPA3 with other MLIPs across different benchmarks. (a) Test energy MAE ($E_{MAE}$, meV/atom) evaluated on the SPICE-MACE-OFF dataset kovacs2023mace. Abbreviations: DES370k M. (DES370k Monomers), DES370k D. (DES370k Dimers) and Sol. AA (Solvated Amino Acids). (b) Torsional energy prediction errors quantified by MAE, RMSE, barrier height MAE (MAEB), and the number of accurately predicted barrier heights within 1 kcal/mol ($\mathrm{NABH}_h$) on the TorsionNet-500 dataset rai2022torsionnet, following evaluation methods described in Ref. yang2024ab. (c) Test energy RMSE ($E_{RMSE}$, meV/H2O) and force RMSE ($F_{RMSE}$, meV/Å) for liquid water and three ice configurations (Ih (b), Ih (c), and Ih (d), sampled at different thermodynamic states) zhang2018deep. (d) Test energy MAE ($E_{MAE}$, meV/atom for the first two datasets and meV for the third) and force MAE ($F_{MAE}$, meV/Å) evaluated on three distinct material datasets from Ref. yin2025alphanet, namely, the Formate Decomposition on Cu (F.D. on Cu), the Defected Bilayer Graphene and the Zeolite dataset. (e) Logarithmic Weighted Average RMSE of energy ($E_{LWARMSE}$, meV/atom) and force ($F_{LWARMSE}$, meV/Å), as defined in Eq. \ref{['eq:LWARMSE']}, evaluated on the DPA2 test sets comprising 18 diverse cases detailed in Ref. zhang2024dpa2.
• Figure 3: LWARMSEs in energy and force predictions evaluated on the DPA2 test sets. The DPA3 models with three layers (DPA3-L3) and six layers (DPA3-L6) were examined at varying LiGS orders $K$.
• Figure 4: Scaling law of the DPA3 models. All evaluations are conducted by measuring validation energy MAEs using models trained on the OMat24 dataset. DPA3 exhibits smooth performance improvement with jointly scaling of (a) model parameters ($N$), (b) training data ($D$), and (c) compute budget ($C$).
• Figure 5: Zero-shot generalizability evaluation of force field prediction tasks across three distinct domains. (a-c) Logarithmic Weighted Average Root Mean Squared Errors (LWARMSE) of energy ($E_{\text{LWARMSE}}$, meV/atom) and force ($F_{\text{LWARMSE}}$, meV/Å), defined by Eq. \ref{['eq:LWARMSE']}, computed separately within each domain. (d) LWARMSE evaluated on the combined set of all 12 datasets across the three domains. Training data summary: DPA-3.1-3M and DPA-2.4-7M zhang2024dpa2peng2025lambench are multitask-pretrained on OpenLAM-v1 openlam-data-v1-webpeng2025lambench. SevenNet-MF-ompa kim2024data is multitask-pretrained on OMat24 barroso2024open, MPtrj deng2023chgnet, and sAlex schmidt2023machine. MACE-MPA-0 batatia2023foundation, Orb-v3 rhodes2025orbv3, and GRACE-2L-OAM bochkarev2024graph are pretrained on OMat24 and subsequently fine-tuned on MPtrj and sAlex. MatterSim-v1-5M yang2024mattersim is trained on a proprietary materials dataset that is not publicly available.