Transfer Learning for Deep Learning-based Prediction of Lattice Thermal Conductivity

TL;DR

This work starts from an existing model (MEGNet~\cite{Chen2019}) and shows that improvements are obtained by fine-tuning a pre-trained version on different tasks, and also shows that a much greater improvement is obtained when first fine-tuning it on a large datasets of low-quality approximations of LTC and then applying a second phase of fine-tuning with high-quality, smaller-scale datasets.

Abstract

Machine learning promises to accelerate the material discovery by enabling high-throughput prediction of desirable macro-properties from atomic-level descriptors or structures. However, the limited data available about precise values of these properties have been a barrier, leading to predictive models with limited precision or the ability to generalize. This is particularly true of lattice thermal conductivity (LTC): existing datasets of precise (ab initio, DFT-based) computed values are limited to a few dozen materials with little variability. Based on such datasets, we study the impact of transfer learning on both the precision and generalizability of a deep learning model (ParAIsite). We start from an existing model (MEGNet~\cite{Chen2019}) and show that improvements are obtained by fine-tuning a pre-trained version on different tasks. Interestingly, we also show that a much greater improvement is obtained when first fine-tuning it on a large datasets of low-quality approximations of LTC (based on the AGL model) and then applying a second phase of fine-tuning with our high-quality, smaller-scale datasets. The promising results obtained pave the way not only towards a greater ability to explore large databases in search of low thermal conductivity materials but also to methods enabling increasingly precise predictions in areas where quality data are rare.

Figures (5)

• Figure 1: The ParAIsite model architecture. Each dense layer contains 350 neurons. The output property is the thermal conductivity (TC).
• Figure 2: Results of validation tests of ParAIsite when using top-performing models cataloged on MatBench Dunn2020 as pre-trained models. As one can see, the MEGNet model Chen2019 shows better stability compared to the CrabNet model Wang2021crabnet on Dataset1. Here, the name of the datasets on which model was pre-trained are indicated inside the box, and the error variablity are shown in red lines.
• Figure 3: Validation loss (MAPE) for models trained and tested on Dataset1 across training epochs.
• Figure 4: Validation loss (MAPE) for models trained and tested on MIX across training epochs.
• Figure 5: Validation loss (MAPE) for models trained and tested on Dataset2 across training epochs.