Materials Representation and Transfer Learning for Multi-Property Prediction

TL;DR

This work tackles the challenge of predicting properties for never-seen material compositions with limited training data. It introduces H-CLMP(T), a hierarchical correlation learning framework that integrates latent embedding learning, pairwise and higher-order property correlations via a multivariate Gaussian and graph attention networks, and generative transfer learning through a conditional Wasserstein GAN trained on MP-DOS data. The approach enables multi-property prediction (optical absorption across 10 energies) for 3-cation metal oxides in 69 unseen spaces, outperforming strong baselines and ablations that remove key components. By leveraging transfer-domain knowledge to augment target-domain inputs, H-CLMP(T) expands the feasible discovery space for materials with tailored optical properties and provides a rigorous benchmark for multi-target regression in materials science.

Abstract

The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements, as well as the relationships among multiple properties, to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates (i) prediction using only a material's composition, (ii) learning and exploitation of correlations among target properties in multi-target regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 3-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data is available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with Transfer learning (H-CLMP(T)) wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well-suited for multi-target regression across the physical sciences.

Figures (7)

• Figure 1: The H-CLMP(T) framework. Components (a) and (b) are jointly-trained parallel models for multi-property prediction and multi-property reconstruction, respectively, where component (b) is a variational autoencoder. The latent representations produced by the encoders are aligned during model training. Each decoder commences with a multivariate Gaussian model for learning pairwise property correlations, where the covariance matrix is shared between components (a) and (b). Higher-order multi-property correlations are learned from the multi-property embeddings via a graph attention network (GAT). Component (a) performs the desired multi-property prediction task, while the multi-property reconstruction of component (b) facilitates training of component (a). Training and deployment of transfer learning is achieved by components (c) and (d), respectively. In (c), a conditional WGAN is trained with transfer domain data and used to construct a transfer representation generator in (d), which augments the model inputs in (a) and (b). Collectively, these components comprise H-CLMP(T) and are implemented by first training model (c) and then using (d) while jointly training (a) and (b). Deployment of the trained model to make new predictions proceeds by evaluating (d) and then (a) for a given composition.
• Figure 2: Predicting optical absorption coefficients in new 3-cation composition spaces. Each setting involves prediction of the absorption coefficient at 10 photon energies for each composition in a 3-cation composition space, where the 3 cation elements never coexist in the training data. a) The baseline model LinInterp interpolates from the perimeter of the composition graph. b) ML models learn from other composition spaces to facilitate predictions. c) Predictions can be further augmented via transfer learning, provided the ML model can capture and exploit the chemical knowledge from the transfer domain.
• Figure 3: Prediction models considered herein that each predict optical absorption in a new 3-cation composition space using only composition as input. LinInterp interpolates each property signal from the compositions subspaces as shown in Figure \ref{['fig:settings']}. ElemNet is pre-trained on the same MP-DOS data used for generative transfer learning, and ElemNetMP is its extension for multi-property prediction. CrabNet and Roost are attention-based models used as provided by the respective publications, which includes pre-training on computational materials data such as formation energy. Each of these models were modified to enable multi-property prediction with a single model (CrabNetMP and RoostMP) and were additionally used with concatenation of the generative transfer learning. H-CLMP and H-CLMP(T) are the hierarchical correlation models of the present work. For each model, the standardized MAE aggregated over 10 photon energies and 69 data instances is shown as a horizontal bar with numeric label.
• Figure 4: The standardized MAE averaged over 69 data instances for each model described in Figure \ref{['fig:models']}, plotted as a function of the 10 photon energy ranges of the source data. H-CLMP and H-CLMP(T) outperform all other models. LinInterp operates in the setting of Figure \ref{['fig:settings']}a by interpolating from the perimeter to the interior of each 3-cation composition triangle. The ML models train using other composition spaces, as shown in Figure \ref{['fig:settings']}b. The models that employ transfer learning (Figure \ref{['fig:settings']}c) are the ElemNet models that pre-train on MP-DOS data as well as CrabNetMP(T), RoostMP(T), and H-CLMP(T) that use generated transfer representation of MP-DOS. By transferring knowledge from the MP-DOS domain, generative transfer learning improves performance of each type of model: CrabNetMP, RoostMP, and H-CLMP, respectively.
• Figure 5: The ground truth and predictions for 4 of the 69 data instances at a single photon energy, 2.54 eV. The prediction models are ordered left-to-right by decreasing aggregate MAE as shown in figure \ref{['fig:models']}, and each composition plot includes a text label of the standardized MAE for the respective model and photon energy. The cation element labels and the color scale for each photon energy are shown for the ground truth data and apply to all models. For all plots of predicted values, only the compositions in the test set are shown, which excludes the perimeter of the composition graph that was used for training. In the top and bottom rows, some 3-cation compositions are missing in each figure due to their absence in the optical absorption data set. Note that the color scales in this figure correspond to the unitless absorption coefficient, which is not directly comparable to the standardized MAE values of Figures \ref{['fig:models']}-\ref{['fig:mae_rand']}.