Graph Transformer Networks for Accurate Band Structure Prediction: An End-to-End Approach

TL;DR

This work addresses the challenge of predicting electronic band structures directly from crystal structures, an end-to-end task not previously solved at scale. It introduces Bandformer, a graph transformer–based framework that encodes crystal structures and translates them into band energies along a continuous high-symmetry k-path, using a graph2seq decoder and an $rFFT$–based frequency representation to efficiently capture band oscillations. Trained on $N_b=6$ near-Fermi bands and a fixed $N_k=128$ path length from $27{,}772$ Materials Project structures, Bandformer achieves a band-energy MAE of $0.304$ eV and derives derived properties such as band gaps with MAEs around $0.205$ eV, demonstrating strong predictive performance and generalizability. This end-to-end approach enables fast electronic-structure predictions for high-throughput materials screening and lays the groundwork for scalable transformer models in band-structure discovery and inverse design.

Abstract

Predicting electronic band structures from crystal structures is crucial for understanding structure-property correlations in materials science. First-principles approaches are accurate but computationally intensive. Recent years, machine learning (ML) has been extensively applied to this field, while existing ML models predominantly focus on band gap predictions or indirect band structure estimation via solving predicted Hamiltonians. An end-to-end model to predict band structure accurately and efficiently is still lacking. Here, we introduce a graph Transformer-based end-to-end approach that directly predicts band structures from crystal structures with high accuracy. Our method leverages the continuity of the k-path and treat continuous bands as a sequence. We demonstrate that our model not only provides accurate band structure predictions but also can derive other properties (such as band gap, band center, and band dispersion) with high accuracy. We verify the model performance on large and diverse datasets.

Figures (4)

• Figure 1: The workflow and architecture of Bandformer.(a) For each input crystal, a crystal graph is constructed from the local environment of atoms in the primitive cell, which is used as the input for the encoder. The high symmetry k-path in the first Brillouin zone is extracted and resampled and used as the input for the decoder graph2seq. (b) Atoms are given element features attached to graph nodes, and interatomic distances are given Gaussian expansions of the distance as features. K-points are given spatial positional encoding. (c) The architecture of Bandformer. (d) The architecture of graph encoder. The input atomic features and the edge features are used to generate a bias term when calculating the attention coefficients in the Biased Multi-head Attention module, which is connected to LayerNorm layer in a ResNet structure he2016. (e) For each k-point along high symmetry k-path, positional encoding of k-point coordinates is used as the initial feature, which is passed to a graph2seq decoder. Within the decoder, k-points learn positional information through Self Attention module. The output is passed to the graph2seq attention module, together with the updated graph encoder output. (f) The workflow of Bandformer. The model predicts real and imaginary components of the FFT-transformed band structure,
• Figure 2: Performance of Bandformer on the test set of Materials Project band structure dataset. Some predicted band structures (colored) vs target band structures (gray) in four quartile of prediction MAE: 0 - 25% (blue), 25 - 50% (orange), 50 - 75% (green), 75 - 100 % (red). The first quartile has MAE between 0.011 and 0.077 eV, the second quartile has MAE between 0.077 and 0.183 eV, the third quartile has MAE between 0.183 and 0.399 eV and the last quartile has MAE between 0.399 and 3.483 eV.
• Figure 3: An example material from test set. The material has formula Ca$_3$BiAs and Materials Project id mp-1013703. (a) The crystal structure of the material. (b) The predicted vs target band structure of the material. (c) The predicted vs target FFT components of the material.
• Figure 4: The prediction vs target plot for multiple properties:(a) constant term of FFT component; (b) FFT real components with $\omega > 0$; (c) FFT imaginary components with $\omega > 0$; (d) band gaps; (e) band centers; (f) band dispersions.