Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter

Zhiyuan Liu, Sihang Li, Yanchen Luo, Hao Fei, Yixin Cao, Kenji Kawaguchi, Xiang Wang, Tat-Seng Chua

TL;DR

MolCA tackles the gap between text-based language models and 2D molecular graph understanding by introducing a cross-modal projector (Q-Former) that maps graph encoder outputs into the LM's text space, enabling open-ended molecule-to-text generation. It employs a three-stage training pipeline—two pretraining stages to align modalities and a LoRA-based fine-tuning stage for efficient downstream adaptation. Empirical results on molecule captioning, IUPAC name prediction, and molecule-text retrieval show state-of-the-art performance, with ablations confirming the value of integrating 2D graphs alongside 1D SMILES. This work paves the way for graph-aware molecular language modeling and potential extensions to 3D modeling and drug discovery while acknowledging dataset limitations and safety considerations.

Abstract

Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA.

MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter

TL;DR

MolCA tackles the gap between text-based language models and 2D molecular graph understanding by introducing a cross-modal projector (Q-Former) that maps graph encoder outputs into the LM's text space, enabling open-ended molecule-to-text generation. It employs a three-stage training pipeline—two pretraining stages to align modalities and a LoRA-based fine-tuning stage for efficient downstream adaptation. Empirical results on molecule captioning, IUPAC name prediction, and molecule-text retrieval show state-of-the-art performance, with ablations confirming the value of integrating 2D graphs alongside 1D SMILES. This work paves the way for graph-aware molecular language modeling and potential extensions to 3D modeling and drug discovery while acknowledging dataset limitations and safety considerations.

Abstract

Language Models (LMs) have demonstrated impressive molecule understanding ability on various 1D text-related tasks. However, they inherently lack 2D graph perception - a critical ability of human professionals in comprehending molecules' topological structures. To bridge this gap, we propose MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter. MolCA enables an LM (e.g., Galactica) to understand both text- and graph-based molecular contents via the cross-modal projector. Specifically, the cross-modal projector is implemented as a Q-Former to connect a graph encoder's representation space and an LM's text space. Further, MolCA employs a uni-modal adapter (i.e., LoRA) for the LM's efficient adaptation to downstream tasks. Unlike previous studies that couple an LM with a graph encoder via cross-modal contrastive learning, MolCA retains the LM's ability of open-ended text generation and augments it with 2D graph information. To showcase its effectiveness, we extensively benchmark MolCA on tasks of molecule captioning, IUPAC name prediction, and molecule-text retrieval, on which MolCA significantly outperforms the baselines. Our codes and checkpoints can be found at https://github.com/acharkq/MolCA.
Paper Structure (17 sections, 6 equations, 7 figures, 11 tables)

This paper contains 17 sections, 6 equations, 7 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Model Architecture
  3. Training Pipeline
  4. Pretrain Stage 1: Learning to Extract Text Relevant Molecule Representations
  5. Pretrain Stage 2: Aligning 2D Molecular Graphs to Texts via Language Modeling
  6. Fine-tune Stage: Uni-Modal Adapter for Efficient Downstream Adaptation
  7. Experiments
  8. Experimental Setting
  9. Molecule Captioning
  10. IUPAC Name Prediction
  11. Molecule-Text Retrieval
  12. Ablation Study on Representation Types
  13. Related Works
  14. Conclusion and Future Works
  15. Complete Related Works
  16. ...and 2 more sections

Figures (7)

  • Figure 1: Comparison of molecular language modeling methods.
  • Figure 2: MolCA's three-stage training pipeline.
  • Figure 3: MolCA's pretrain stage 1. The graph encoder and the cross-modal projector (i.e., Q-Former) are jointly optimized using three cross-modal tasks. Modules of the same color share weights.
  • Figure 4: MolCA's pretrain stage 2 by molecule captioning.
  • Figure 5: MolCA's fine-tune stage for molecule-to-text generation. The example shows the prediction of a molecule's IUPAC name.
  • ...and 2 more figures