Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Motif-aware Attribute Masking for Molecular Graph Pre-training

Eric Inae, Gang Liu, Meng Jiang

TL;DR

Motif-aware Attribute Masking for Molecular Graph Pre-training tackles the limitation of random node masking by promoting inter-motif knowledge transfer in molecular graphs. The authors introduce MoAMa, a motif-aware masking framework that masks entire motifs and reconstructs their node attributes, aided by a knowledge-enhanced auxiliary loss based on Tanimoto similarity to align latent representations with chemical space structure. Empirical results on 11 MoleculeNet tasks show MoAMa delivering consistent improvements over baselines in both classification and regression, with ablation analyses highlighting the benefit of the auxiliary loss for capturing global motif information. The work provides a quantitative framework for assessing inter-motif influence and demonstrates the practical value of motif-aware pre-training for molecular property prediction.

Abstract

Attribute reconstruction is used to predict node or edge features in the pre-training of graph neural networks. Given a large number of molecules, they learn to capture structural knowledge, which is transferable for various downstream property prediction tasks and vital in chemistry, biomedicine, and material science. Previous strategies that randomly select nodes to do attribute masking leverage the information of local neighbors However, the over-reliance of these neighbors inhibits the model's ability to learn from higher-level substructures. For example, the model would learn little from predicting three carbon atoms in a benzene ring based on the other three but could learn more from the inter-connections between the functional groups, or called chemical motifs. In this work, we propose and investigate motif-aware attribute masking strategies to capture inter-motif structures by leveraging the information of atoms in neighboring motifs. Once each graph is decomposed into disjoint motifs, the features for every node within a sample motif are masked. The graph decoder then predicts the masked features of each node within the motif for reconstruction. We evaluate our approach on eight molecular property prediction datasets and demonstrate its advantages.

Motif-aware Attribute Masking for Molecular Graph Pre-training

TL;DR

Motif-aware Attribute Masking for Molecular Graph Pre-training tackles the limitation of random node masking by promoting inter-motif knowledge transfer in molecular graphs. The authors introduce MoAMa, a motif-aware masking framework that masks entire motifs and reconstructs their node attributes, aided by a knowledge-enhanced auxiliary loss based on Tanimoto similarity to align latent representations with chemical space structure. Empirical results on 11 MoleculeNet tasks show MoAMa delivering consistent improvements over baselines in both classification and regression, with ablation analyses highlighting the benefit of the auxiliary loss for capturing global motif information. The work provides a quantitative framework for assessing inter-motif influence and demonstrates the practical value of motif-aware pre-training for molecular property prediction.

Abstract

Attribute reconstruction is used to predict node or edge features in the pre-training of graph neural networks. Given a large number of molecules, they learn to capture structural knowledge, which is transferable for various downstream property prediction tasks and vital in chemistry, biomedicine, and material science. Previous strategies that randomly select nodes to do attribute masking leverage the information of local neighbors However, the over-reliance of these neighbors inhibits the model's ability to learn from higher-level substructures. For example, the model would learn little from predicting three carbon atoms in a benzene ring based on the other three but could learn more from the inter-connections between the functional groups, or called chemical motifs. In this work, we propose and investigate motif-aware attribute masking strategies to capture inter-motif structures by leveraging the information of atoms in neighboring motifs. Once each graph is decomposed into disjoint motifs, the features for every node within a sample motif are masked. The graph decoder then predicts the masked features of each node within the motif for reconstruction. We evaluate our approach on eight molecular property prediction datasets and demonstrate its advantages.
Paper Structure (22 sections, 12 equations, 2 figures, 7 tables, 1 algorithm)

This paper contains 22 sections, 12 equations, 2 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Inter-Motif Influence
  2. Proposed Solution
  3. Motif-aware Attribute Masking and Reconstruction
  4. Experiments
  5. Experimental Settings
  6. Datasets
  7. Validation methods and evaluation metrics
  8. Model configurations
  9. Baselines
  10. Results
  11. Inter-motif Influence Analysis
  12. Evaluation Complexity
  13. Conclusions
  14. Appendix
  15. Inter-motif Influence
  16. ...and 7 more sections

Figures (2)

  • Figure 2: InfRatio and MRR measurements of pre-trained models against average AUC-ROC scores. Each point represents a different pre-trained model, with $\Diamondblack$ denoting MoAMa. The InfRatio and MRR measurements demonstrate strong positive/negative correlations. These trends reveal performance can be correlated with inter-motif knowledge transfer for attribute masking strategies.
  • Figure 3: Inter-motif knowledge transfer score by motif count. A higher $\text{MRR}_{\text{inter}}^{(n)}$ score denotes greater inter-motif knowledge transfer.