Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models

Zhenzhong Wang, Zehui Lin, Wanyu Lin, Ming Yang, Minggang Zeng, Kay Chen Tan

TL;DR

Lamole addresses the need for chemically meaningful explanations in molecular property prediction by using Group SELFIES and an information-flow framework that fuses attention and gradients. A marginal loss aligns explanations with chemist-annotated substructures, with theoretical grounding in the tangent space of the data manifold, yielding concept-aligned explanations. Empirically, Lamole achieves predictive accuracy comparable to state-of-the-art baselines while boosting explanation quality across toxicity and ADME tasks, and demonstrates actionable molecular editing guided by interpretable fragments. This combination offers a practical path from prediction to rational molecular design, with potential extension to broader chemotypes and properties in future work.

Abstract

Providing explainable molecular property predictions is critical for many scientific domains, such as drug discovery and material science. Though transformer-based language models have shown great potential in accurate molecular property prediction, they neither provide chemically meaningful explanations nor faithfully reveal the molecular structure-property relationships. In this work, we develop a framework for explainable molecular property prediction based on language models, dubbed as Lamole, which can provide chemical concepts-aligned explanations. We take a string-based molecular representation -- Group SELFIES -- as input tokens to pretrain and fine-tune our Lamole, as it provides chemically meaningful semantics. By disentangling the information flows of Lamole, we propose combining self-attention weights and gradients for better quantification of each chemically meaningful substructure's impact on the model's output. To make the explanations more faithfully respect the structure-property relationship, we then carefully craft a marginal loss to explicitly optimize the explanations to be able to align with the chemists' annotations. We bridge the manifold hypothesis with the elaborated marginal loss to prove that the loss can align the explanations with the tangent space of the data manifold, leading to concept-aligned explanations. Experimental results over six mutagenicity datasets and one hepatotoxicity dataset demonstrate Lamole can achieve comparable classification accuracy and boost the explanation accuracy by up to 14.3%, being the state-of-the-art in explainable molecular property prediction.

Explainable Molecular Property Prediction: Aligning Chemical Concepts with Predictions via Language Models

TL;DR

Lamole addresses the need for chemically meaningful explanations in molecular property prediction by using Group SELFIES and an information-flow framework that fuses attention and gradients. A marginal loss aligns explanations with chemist-annotated substructures, with theoretical grounding in the tangent space of the data manifold, yielding concept-aligned explanations. Empirically, Lamole achieves predictive accuracy comparable to state-of-the-art baselines while boosting explanation quality across toxicity and ADME tasks, and demonstrates actionable molecular editing guided by interpretable fragments. This combination offers a practical path from prediction to rational molecular design, with potential extension to broader chemotypes and properties in future work.

Abstract

Providing explainable molecular property predictions is critical for many scientific domains, such as drug discovery and material science. Though transformer-based language models have shown great potential in accurate molecular property prediction, they neither provide chemically meaningful explanations nor faithfully reveal the molecular structure-property relationships. In this work, we develop a framework for explainable molecular property prediction based on language models, dubbed as Lamole, which can provide chemical concepts-aligned explanations. We take a string-based molecular representation -- Group SELFIES -- as input tokens to pretrain and fine-tune our Lamole, as it provides chemically meaningful semantics. By disentangling the information flows of Lamole, we propose combining self-attention weights and gradients for better quantification of each chemically meaningful substructure's impact on the model's output. To make the explanations more faithfully respect the structure-property relationship, we then carefully craft a marginal loss to explicitly optimize the explanations to be able to align with the chemists' annotations. We bridge the manifold hypothesis with the elaborated marginal loss to prove that the loss can align the explanations with the tangent space of the data manifold, leading to concept-aligned explanations. Experimental results over six mutagenicity datasets and one hepatotoxicity dataset demonstrate Lamole can achieve comparable classification accuracy and boost the explanation accuracy by up to 14.3%, being the state-of-the-art in explainable molecular property prediction.
Paper Structure (13 sections, 1 theorem, 8 equations, 12 figures, 9 tables)

This paper contains 13 sections, 1 theorem, 8 equations, 12 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Our Design: Lamole
  5. Theoretical Analysis
  6. Experiments
  7. Experimental Setup
  8. Results and Discussion
  9. Applicability to Broader Properties
  10. Ablation Studies
  11. Computational Cost
  12. Case Study: Actionable Utility for Molecular Editing
  13. Conclusions

Key Result

Theorem 1

The marginal loss of Eq. (eq:l1) aligns the gradient-based explanations $\nabla_g \log p(y|g)$ with the tangent space of the causal feature manifold $\mathcal{M}_c$, thus respecting the structure-property relationships.

Figures (12)

  • Figure 1: (a) The molecule visualization of prediction/explanation. The interaction between the benzene ring and the nitro group (highlighted in red) induces the mutagenic property of the molecule. (b)-(e) are the explanation results obtained with various methods: (b) OrphicX lin2022orphicx; (c) GNNExplainer gnnexplainer, (d) GNN with gradient-based explainability technique (GradCAM Selvaraju_2017_ICCV); (e) Bert with GradCAM (molecular string SMILES as input); (f) Bert with GradCAM (molecular string Group SELFIES cheng2022group as the input representation); (g) Our method Lamole assigns an importance score to each functional group/fragment to indicate their contribution to the property.
  • Figure 2: The SMILES and Group SELFIES strings of p-nitrobenzoic acid molecule ($\mathrm{C_7H_5NO_4}$): The tokens in the Group SELFIES string highlighted by color are the corresponding functional groups.
  • Figure 3: An illustration of Lamole. Left panel: Group SELFIES strings are tokenized for fine-tuning the pre-trained language model, and an MLP classifier is equipped with a cross-entropy loss $\mathcal{L}_{CE}$ for molecular property prediction. Right panel: We disentangle the information flows of the Transformer to assert that both attention weights and gradient determine the output. Therefore, we incorporate the attention weights and gradients together to generate importance scores $\mathbf{v}$ as explanations. In addition, a marginal loss $\mathcal{L}_{M}$ is designed to align explanations with the chemists’ annotations $\mathbf{m}$.
  • Figure 4: Twelve ground truth substructures of the Liver dataset. Lowercase element symbols represent aromatic atoms of the element; the letter "a" matches any aromatic atom. Elements in square brackets match any of the elements in a molecule.
  • Figure 5: The explanation accuracy of Lamole with different annotation rates on the Mutag, PTC-FM, and PTC-MR datasets.
  • ...and 7 more figures

Theorems & Definitions (3)

  • Definition 1
  • Definition 2
  • Theorem 1