KappaFormer: Physics-aware Transformer for lattice thermal conductivity via cross-domain transfer learning

Abstract

Machine learning has been widely used for predicting material properties. However, efficient prediction of lattice thermal conductivity ($κ_\mathrm{L}$) remains a long-standing challenge, primarily due to the scarcity of high-quality training data. Here we introduce KappaFormer, a physics-aware Transformer architecture that embeds the harmonic-anharmonic decomposition of $κ_\mathrm{L}$ within the network. KappaFormer comprises a harmonic branch pre-trained on large-scale elastic property data and an anharmonic branch fine-tuned on limited experimental $κ_\mathrm{L}$ data, enabling effective knowledge transfer and enhanced generalization. High-throughput screening with KappaFormer identifies multiple candidates with ultralow $κ_\mathrm{L}$, which are further confirmed by first-principles calculations. Physics interpretability further elucidates the vibrational mechanisms governing thermal transport suppression, linking structural motifs to strong anharmonicity. This study provides a generalizable framework for physics-guided machine learning to accelerate the discovery of new materials.

Figures (6)

• Figure 1: Architecture of KappaFormer. a) Overview of the proposed framework that embeds the harmonic–anharmonic decomposition of $\kappa_\mathrm{L}$ within a graph-based attention network. b) Graph embedding. c) Atom-pair-aware graph attention. d) Scalar recurrent block. e) Multi-gate Mixture-of-Experts. f) Physics formulation.
• Figure 2: Model training and performance. a) Dual-stage cross-domain transfer learning strategy to overcome data-scarce conditions. b)--d) ML-predicted values from KappaFormer compared with reference values for B, G, and $\kappa_\mathrm{L}$. The blue and orange data points denote the training and test set, respectively. The insets show the cumulative kernel density estimation (KDE) plots of the relative errors on the test sets. e)--g) t-SNE visualization of the hidden embeddings for B, G, and $\kappa_\mathrm{L}$ in the latent space. Each point indicates a distinct crystal structure.
• Figure 3: High-throughput material discovery. a) Periodic table trends of elements associated with low $\kappa_\mathrm{L}$ materials, with the colour bar indicating the percentage probability of each element. Inset shows the histogram distribution of $\kappa_\mathrm{L}$ for the training data and newly discoverd materials with low $\kappa_\mathrm{L}$, where each dataset is normalized independently to sum to 100 percent. b) Scatter distribution of predicted B and G in the prediction dataset, colored by $\log_{10} \kappa_\mathrm{L}$. Inset highlights the low-modulus region and representative discovered materials with ultralow $\kappa_\mathrm{L}$, i.e., CsNb$_2$Br$_9$, Cs$_2$AgI$_3$, and Cs$_6$CdSe$_4$. c) Bar charts comparing DFT results with model predictions with (w/) and without (w/o) physics for the three materials. d) Temperature dependence of the DFT-calculated $\kappa_\mathrm{L}$ along different axes for the three materials.
• Figure 4: Structure characteristics and phonon thermal transport properties of the discovered three materials with ultralow $\kappa_\mathrm{L}$. Crystal structures and the projected 2D ELF diagrams of a) CsNb$_2$Br$_9$, b) Cs$_2$AgI$_3$, and c) Cs$_6$CdSe$_4$. Phonon dispersion (left panels) and spectral $\kappa_\mathrm{L}(\omega)$ (right panels) of d) CsNb$_2$Br$_9$, e) Cs$_2$AgI$_3$, and f) Cs$_6$CdSe$_4$.
• Figure 5: Physics interpretability of KappaFormer. The whole framework including deriving atom-resolved contributions from KappaFormer and elucidating phonon thermal transport mechanisms from DFT calculations.