LeMat-Bulk: aggregating, and de-duplicating quantum chemistry materials databases

TL;DR

LeMat-Bulk tackles data fragmentation across major quantum chemistry databases by unifying MP, OQMD, and Alexandria into a single, standardized dataset. It introduces BAWL, a graph based hashing fingerprint that encodes bonding graphs and composition to enable fast, scalable de-duplication and cross database matching, with Short-BAWL offering a more robust variant. The work benchmarks these fingerprints against existing structure similarity and fingerprint methods, demonstrates robust performance under perturbations and in disordered systems, and enables cross functional comparisons among PBE, PBESol, and SCAN to reveal systematic trends in energy and magnetization. Overall, LeMat-Bulk provides a scalable framework for interoperable, reproducible, and searchable materials data that supports data-driven discovery and closed-loop materials design while advancing benchmarking for structure novelty and de-duplication.

Abstract

The rapid expansion of materials science databases has driven machine learning-based discovery while also posing challenges in data integration, duplication, and interoperability. Robust standardization and de-duplication methods are needed to address these issues and streamline materials research. We present LeMat-Bulk, a unified dataset combining Materials Project, OQMD, and Alexandria, encompassing over 5.3 million PBE-calculated materials and also representing the largest collection of PBESol and SCAN functional calculations. Our methodology standardizes calculations across databases that utilize different parameters, effectively addressing redundancy and enhancing cross-compatibility. To de-duplicate, we propose a hashing function which we termed the Bonding Algorithm Weisfeiller-Lehman (BAWL). We comprehensively benchmark this fingerprint under atomic noise, lattice strain, and symmetry transformations, demonstrating that it outperforms existing fingerprinting techniques such as SLICES, and CLOUD in robustness while offering greater computational efficiency than similarity-based approaches such as Pymatgen's StructureMatcher. Additionally, the fingerprint facilitates the analysis of functional-dependent trends (PBE, PBESol, SCAN) offering a scalable framework for data-driven materials science.

Figures (21)

• Figure 1: Illustration of the BAWL hashing method. A graph representation of the crystal structure is constructed using the ECoN algorithm. The graph is hashed using Weisfeiller-Lehman. This hash is concatenated with the spacegroup info from Spglib and the reduced chemical formula
• Figure 2: 2D block chart demonstrating chemical bias in LeMat-Bulk. Less bias is present in LeMat-Bulk compared to any single database.
• Figure 3: Distribution of above the hull materials across all materials in LeMat-Bulk by database of origin. Pie chart showcase percent of materials originating from each database that are on the hull, potentially metastable and unstable.
• Figure 4: Benchmark results of various fingerprint and similarity methods across noise on atomic coordinates, noise on lattice vectors, translation, strain and crystal symmetry operations. Inclusion of DFT sensitivity based on DFT relaxation tests. Thickness around lines indicates areas of higher variability across the methods tested.
• Figure 5: