Enhancing material property prediction with ensemble deep graph convolutional networks

TL;DR

This work investigates how ensemble strategies can enhance material property predictions from graph-based neural networks. By training CGCNN and MT-CGCNN across multiple epochs and selecting top-performing models, the authors compare prediction-based and model-based ensembling, finding that averaging predictions from top models yields robust improvements in $ΔE^{f}$, $E_g$, and $ρ$ on 33{,}990 stable inorganic materials from the Materials Project. The results demonstrate that the loss landscape harbors multiple informative regions, and that a simple ensemble built from these regions can surpass the single best validation model in most scenarios. The approach offers a practical route to more accurate and generalizable materials-property predictions with modest computational cost, with potential impact on accelerated materials discovery and design.

Abstract

Machine learning (ML) models have emerged as powerful tools for accelerating materials discovery and design by enabling accurate predictions of properties from compositional and structural data. These capabilities are vital for developing advanced technologies across fields such as energy, electronics, and biomedicine, potentially reducing the time and resources needed for new material exploration and promoting rapid innovation cycles. Recent efforts have focused on employing advanced ML algorithms, including deep learning - based graph neural network, for property prediction. Additionally, ensemble models have proven to enhance the generalizability and robustness of ML and DL. However, the use of such ensemble strategies in deep graph networks for material property prediction remains underexplored. Our research provides an in-depth evaluation of ensemble strategies in deep learning - based graph neural network, specifically targeting material property prediction tasks. By testing the Crystal Graph Convolutional Neural Network (CGCNN) and its multitask version, MT-CGCNN, we demonstrated that ensemble techniques, especially prediction averaging, substantially improve precision beyond traditional metrics for key properties like formation energy per atom ($ΔE^{f}$), band gap ($E_{g}$) and density ($ρ$) in 33,990 stable inorganic materials. These findings support the broader application of ensemble methods to enhance predictive accuracy in the field.

Figures (6)

• Figure 1: Overview of GCNN in material property prediction tasks. Initially, the crystal structure is created from information files. Then, the crystal graph is constructed from the structure. GCN layers and pooling layers are utilized to obtain crystal embeddings, after which fully connected (FC) layers are employed to predict properties. For a single-task head, a single FC layer is used, while for the prediction of multiple properties, multiple FC layers are utilized.
• Figure 2: Overview of ensemble strategies: (a) prediction averaging ensemble technique and (b) the model averaging ensemble technique.
• Figure 3: Results for Prediction Ensemble on Single-task CGCNN and Mulit-task CGCNN.
• Figure 4: Prediction ensemble CGCNN$_{\text{1}}$ across different groups of data distribution for three properties.
• Figure 5: Prediction ensemble MT-CGCNN$_{\text{1}}$ across different groups of data distribution for three properties.