Understanding the Capabilities of Molecular Graph Neural Networks in Materials Science Through Multimodal Learning and Physical Context Encoding

TL;DR

This work addresses the limitations of chemistry-aware molecular GNNs that rely solely on XYZ geometry by introducing a multimodal framework that incorporates PubChem-derived textual descriptors. A CLIP-based transformer encodes textual context (e.g., IUPAC names, formulas, spectral data) into $oldsymbol{t} \\in \\mathbb{R}^{768}$, which is fused with geometry embeddings $oldsymbol{g} \\in \\mathbb{R}^n$ from equivariant GNNs through a gated mechanism to produce a unified representation for property prediction. Across QM9 augmented with textual data, four SOTA architectures (SchNet, DimeNet++, Equiformer, FAENet) show improved MAE for dipole moment, HOMO energy, and the HOMO–LUMO gap, but gains are inconsistent for polarizability, ZPVE, and energetic properties, indicating limits of generic textual descriptors. The results suggest that while textual context can augment electronic-property predictions, the learned representations across architectures converge on similar patterns, highlighting the need for more expressive descriptors and refined fusion strategies to fully leverage multimodal information. Overall, the study demonstrates the value and current constraints of integrating chemical metadata with geometric information to advance materials discovery, while outlining practical mitigation strategies for negative transfer and gradient conflicts.

Abstract

Molecular graph neural networks (GNNs) often focus exclusively on XYZ-based geometric representations and thus overlook valuable chemical context available in public databases like PubChem. This work introduces a multimodal framework that integrates textual descriptors, such as IUPAC names, molecular formulas, physicochemical properties, and synonyms, alongside molecular graphs. A gated fusion mechanism balances geometric and textual features, allowing models to exploit complementary information. Experiments on benchmark datasets indicate that adding textual data yields notable improvements for certain electronic properties, while gains remain limited for others. Furthermore, the GNN architectures display similar performance patterns (improving and deteriorating on analogous targets), suggesting they learn comparable representations rather than distinctly different physical insights.

Figures (2)

• Figure 1: Comparison of common SOTA approaches and the proposed approach. The proposed approach is able to utilize multiple chemical databases for different property annotations of materials as an additional modality, whereas the common approaches rely solely on XYZ files, which consist of element names and their 3D coordinates.
• Figure 2: The proposed approach integrates textual descriptions retrieved from the PubChem library with the XYZ structural data provided by the QM9 dataset. Textual descriptions are processed by a transformer model to generate textual embeddings, while XYZ coordinates are input into a GNN to derive molecular embeddings. Subsequently, these embeddings are fused into a unified representation, which is then used to predict molecular properties.