CrysToGraph: A Comprehensive Predictive Model for Crystal Materials Properties and the Benchmark

Abstract

The ionic bonding across the lattice and ordered microscopic structures endow crystals with unique symmetry and determine their macroscopic properties. Unconventional crystals, in particular, exhibit non-traditional lattice structures or possess exotic physical properties, making them intriguing subjects for investigation. Therefore, to accurately predict the physical and chemical properties of crystals, it is crucial to consider long-range orders. While GNN excels at capturing the local environment of atoms in crystals, they often face challenges in effectively capturing longer-ranged interactions due to their limited depth. In this paper, we propose CrysToGraph ($\textbf{Crys}$tals with $\textbf{T}$ransformers $\textbf{o}$n $\textbf{Graph}$s), a novel transformer-based geometric graph network designed specifically for unconventional crystalline systems, and UnconvBench, a comprehensive benchmark to evaluate models' predictive performance on unconventional crystal materials such as defected crystals, low-dimension crystals and MOF. CrysToGraph effectively captures short-range interactions with transformer-based graph convolution blocks as well as long-range interactions with graph-wise transformer blocks. CrysToGraph proofs its effectiveness in modelling unconventional crystal materials in multiple tasks, and moreover, it outperforms most existing methods, achieving new state-of-the-art results on the benchmarks of both unconventional crystals and traditional crystals.

Figures (24)

• Figure 1: An overview of the architecture of CrysToGraph. In this paper, $\mathcal{G}$ denotes the original crystal graph, $L(\mathcal{G})$ denotes the line graph built upon the edges of the direct crystal graphs. For inputs, $x_i$ denotes the atom (node) feature of node $i$, $e_{ji}$ represents the bond (edge) feature of edge $(j,i)$, $t_{jik}$ represents the edge feature in the line graph, also the relationship between edge $(j,i)$ and $(i,k)$, $p_i$ denotes the positional encoding on atom $i$. Details of the graphs and positional encoding can be found in section 3.1 and 3.2. Details of the architecture can be found in section 3.3.
• Figure 2: Left: A hexagonal close-packed ($hcp$) structure with atoms stacked in layers in an abab… pattern into a bulk crystal. Right: The theoretical maximum number of neighbors for a single atom (illustrated in orange) is 12.
• Figure 3: Construction of line graph $L(\mathcal{G})$ from direct graph $\mathcal{G}$. The edges in direct graph $\mathcal{G}$ are considered as nodes in line graph $L\mathcal{G}$, and the angles of two edges in $\mathcal{G}$ are constructed as the edges in $L\mathcal{G}$.
• Figure 4: Structure of an eTGC layer. An eTGC block consists of two eTGC layers that take different inputs.
• Figure 5: Structure of multi-head neighbor attention in eTGC layers.