Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

Namkyeong Lee, Siddhartha Laghuvarapu, Chanyoung Park, Jimeng Sun

TL;DR

The paper tackles the two main bottlenecks of Molecule Language Models: limited molecule-text paired data and uneven expert coverage. It introduces AMOLE, which augments molecule-text pairs through structural similarity preserving text sharing and transfers expert knowledge via an Expertise Reconstruction loss, enabling richer cross-modal representations. Through extensive experiments across zero-shot cross-modal retrieval, zero-shot QA, molecular property prediction, and zero-shot virtual screening, AMOLE demonstrates consistent improvements and practical potential for drug discovery. The work also shows AMOLE's applicability to architectures beyond CLIP-style models and discusses directions to further expand descriptor coverage using decoder-based generation and style transfer.

Abstract

Recently, there has been a growing interest among researchers in understanding molecules and their textual descriptions through molecule language models (MoLM). However, despite some early promising developments, the advancement of MoLM still trails significantly behind that of vision language models (VLM). This is because unique challenges exist apart from VLM in the field of MoLM due to 1) a limited amount of molecule-text paired data and 2) missing expertise that occurred due to the specialized areas of focus among the experts. To this end, we propose AMOLE, which 1) augments molecule-text pairs with structural similarity preserving loss, and 2) transfers the expertise between the molecules. Specifically, AMOLE enriches molecule-text pairs by sharing descriptions among structurally similar molecules with a novel structural similarity preserving loss. Moreover, we propose an expertise reconstruction loss to transfer knowledge from molecules that have extensive expertise to those with less expertise. Extensive experiments on various downstream tasks demonstrate the superiority of AMOLE in comprehending molecules and their descriptions, highlighting its potential for application in real-world drug discovery. The source code for AMOLE is available at https://github.com/Namkyeong/AMOLE.

Vision Language Model is NOT All You Need: Augmentation Strategies for Molecule Language Models

TL;DR

The paper tackles the two main bottlenecks of Molecule Language Models: limited molecule-text paired data and uneven expert coverage. It introduces AMOLE, which augments molecule-text pairs through structural similarity preserving text sharing and transfers expert knowledge via an Expertise Reconstruction loss, enabling richer cross-modal representations. Through extensive experiments across zero-shot cross-modal retrieval, zero-shot QA, molecular property prediction, and zero-shot virtual screening, AMOLE demonstrates consistent improvements and practical potential for drug discovery. The work also shows AMOLE's applicability to architectures beyond CLIP-style models and discusses directions to further expand descriptor coverage using decoder-based generation and style transfer.

Abstract

Recently, there has been a growing interest among researchers in understanding molecules and their textual descriptions through molecule language models (MoLM). However, despite some early promising developments, the advancement of MoLM still trails significantly behind that of vision language models (VLM). This is because unique challenges exist apart from VLM in the field of MoLM due to 1) a limited amount of molecule-text paired data and 2) missing expertise that occurred due to the specialized areas of focus among the experts. To this end, we propose AMOLE, which 1) augments molecule-text pairs with structural similarity preserving loss, and 2) transfers the expertise between the molecules. Specifically, AMOLE enriches molecule-text pairs by sharing descriptions among structurally similar molecules with a novel structural similarity preserving loss. Moreover, we propose an expertise reconstruction loss to transfer knowledge from molecules that have extensive expertise to those with less expertise. Extensive experiments on various downstream tasks demonstrate the superiority of AMOLE in comprehending molecules and their descriptions, highlighting its potential for application in real-world drug discovery. The source code for AMOLE is available at https://github.com/Namkyeong/AMOLE.
Paper Structure (30 sections, 7 equations, 9 figures, 7 tables)

This paper contains 30 sections, 7 equations, 9 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Problem Statement
  5. Tanimoto Similarity
  6. Molecule-Text Contrastive Learning
  7. Methodology
  8. Augmenting Molecule-Text pairs
  9. Structural Similarity Preserving Loss
  10. Expertise Transfer Module
  11. Model Training
  12. Experiments
  13. Experimental Setup
  14. Zero-Shot Cross-Modal Retrieval
  15. Zero-Shot Question and Answering
  16. ...and 15 more sections

Figures (9)

  • Figure 1: (a) Rivastigmine's textual descriptions from various experts. (b) The majority of molecules in the PubChem database have only one description provided by an expert.
  • Figure 2: Overall model architecture of AMOLE.
  • Figure 3: Sensitivity analysis on $k$.
  • Figure 4: Hit rate (%) in zero-shot virtual screening task.
  • Figure 5: Sensitivity analysis on $\alpha$.
  • ...and 4 more figures