Implicit Delta Learning of High Fidelity Neural Network Potentials

TL;DR

IDLe introduces an end-to-end multi-task framework that leverages cheaper low-fidelity QM data to implicitly learn high-fidelity neural network potentials, drastically reducing HF data requirements while preserving accuracy and inference costs. By sharing a latent representation across fidelity-specific heads, IDLe decodes $E^{HF}$ from $E^{LF}$ labels, enabling robust IID generalization and improved chemical coverage. Across multiple QM datasets and distribution shifts, IDLe achieves chemical accuracy with orders of magnitude fewer HF labels, and demonstrates substantial computational savings (up to ~25x HF data and CPU-time reductions) in both IID and out-of-distribution settings. The work also contributes a large LF-label dataset (~11 million points) to support future multi-fidelity NNP research, with broader implications for efficient MD simulations in materials science and drug discovery.

Abstract

Neural network potentials (NNPs) offer a fast and accurate alternative to ab-initio methods for molecular dynamics (MD) simulations but are hindered by the high cost of training data from high-fidelity Quantum Mechanics (QM) methods. Our work introduces the Implicit Delta Learning (IDLe) method, which reduces the need for high-fidelity QM data by leveraging cheaper semi-empirical QM computations without compromising NNP accuracy or inference cost. IDLe employs an end-to-end multi-task architecture with fidelity-specific heads that decode energies based on a shared latent representation of the input atomistic system. In various settings, IDLe achieves the same accuracy as single high-fidelity baselines while using up to 50x less high-fidelity data. This result could significantly reduce data generation cost and consequently enhance accuracy and generalization, and expand chemical coverage for NNPs, advancing MD simulations for material science and drug discovery. Additionally, we provide a novel set of 11 million semi-empirical QM calculations to support future multi-fidelity NNP modeling.

Figures (9)

• Figure 1: IID performance with DFT as HF labels. We compare of MAE of IDLe with several baselines for a varying amount of DFT labels on QMugs (left) and QM7-X (right).
• Figure 2: IID performance with CCSD(T) as HF labels: MAE of IDLE compared to several baselines for a varying amount of CCSD(T) labels on ANI1-ccxvL.
• Figure 3: OOD performance on SpiceV2->1 with $\omega$B97M-D3(BJ)/def2-TZVPPD as HF labels: On the left, we show the average MAE of IDLe compared to other methods for a varying amount of DFT labels (without the PubChem-Boron-Silicon subset). On the right, we distinguish between all the subsets of SpiceV1->2 with $0\%$ and $1\%$ of the HF labels available during training.
• Figure 4: Extrapolation to larger molecules. MAE on datasets $B$ for three different distribution shifts with $(n_A=60, n_B=61)$, $(n_A=40, n_B=80)$ and $(n_A=30, n_B=120)$.
• Figure 5: Neural scaling of IDLe. Mean absolute error (MAE) and exponent $\beta$ on QMugsvL when varying the amount of DFT data for a fixed amount of LF GFN2-xTB data (left) and vice versa (right).