Scalable Diffusion for Materials Generation

TL;DR

The paper addresses scalable discovery of stable crystal materials by introducing UniMat, a periodic-table–aligned unified crystal representation, and applying diffusion models to generate both unconditional and composition-conditioned crystal structures. By representing atoms as a 4D tensor and employing interleaved attention and convolution layers, UniMat scales to large, complex systems while preserving chemical inductive biases. The authors validate generated materials with rigorous DFT-based metrics, showing UniMat yields lower formation energies and more hull-stable candidates than prior graph-based methods, and demonstrate zero-shot conditional generation that accelerates structure discovery relative to AIRSS. This work enables data-driven, scalable materials exploration with physics-backed verification, potentially accelerating discovery of novel stable materials at industrial scales.

Abstract

Generative models trained on internet-scale data are capable of generating novel and realistic texts, images, and videos. A natural next question is whether these models can advance science, for example by generating novel stable materials. Traditionally, models with explicit structures (e.g., graphs) have been used in modeling structural relationships in scientific data (e.g., atoms and bonds in crystals), but generating structures can be difficult to scale to large and complex systems. Another challenge in generating materials is the mismatch between standard generative modeling metrics and downstream applications. For instance, common metrics such as the reconstruction error do not correlate well with the downstream goal of discovering stable materials. In this work, we tackle the scalability challenge by developing a unified crystal representation that can represent any crystal structure (UniMat), followed by training a diffusion probabilistic model on these UniMat representations. Our empirical results suggest that despite the lack of explicit structure modeling, UniMat can generate high fidelity crystal structures from larger and more complex chemical systems, outperforming previous graph-based approaches under various generative modeling metrics. To better connect the generation quality of materials to downstream applications, such as discovering novel stable materials, we propose additional metrics for evaluating generative models of materials, including per-composition formation energy and stability with respect to convex hulls through decomposition energy from Density Function Theory (DFT). Lastly, we show that conditional generation with UniMat can scale to previously established crystal datasets with up to millions of crystals structures, outperforming random structure search (the current leading method for structure discovery) in discovering new stable materials.

Figures (8)

• Figure 1: UniMat representation of crystal structures. Crystals are represented by the atom locations stored at the corresponding elements in the periodic table (and additional unit cell parameters if coordinates are fractional). For instance, the bottom right atom Na in the crystal is located at $[0.5, 0, 0]$, hence the periodic table has value $[0.5, 0, 0]$ at the Na entry. Note the structure on the left is only showing 1/8 of the unit cell.
• Figure 2: Illustration of the denoising process for unconditional generation with UniMat. The denoising model learns to move atoms from random locations back to their original locations. Atoms not present in the crystal are moved to the null location during the denoising process, allowing crystals with an arbitrary number of atoms to be generated.
• Figure 3: Qualitative evaluation of materials generated by CDVAE xie2021crystal (left) and UniMat (right) trained on MP-20 in comparison to the test set materials of the same composition. Materials generated by UniMat generally align better with the test set.
• Figure 4: UniMat trained with a larger feature dimension results in better validity and coverage.
• Figure 5: Difference in $E_f$ for each composition generated by UniMat and CDVAE, i.e., $E_{f, x}^A - E_{f, x'}^B$, where $A$ and $B$ are sets of structures generated by UniMat and CDVAE, respectively. UniMat generates more structures with lower $E_f$.