Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model

Dongki Kim, Wonbin Lee, Sung Ju Hwang

TL;DR

Mol-LLaMA introduces a general-purpose molecular language model that learns broad molecular knowledge with explainability and reasoning. It couples two molecular encoders (2D MoleculeSTM and 3D UniMol) through a 2D-3D blending module and a Q-Former projector to an LLM, all trained via a two-stage process: molecular representation learning and end-to-end instruction tuning with a large, domain-specific dataset (Mol-LLaMA-Instruct) containing 284k high-quality samples. The approach yields superior performance on general molecular understanding, molecular property prediction (PAMPA/BBP), and the MoleculeQA benchmark, outperforming prior molecular LLMs and baselines while enabling diverse prompt types and few-shot capabilities. The work demonstrates the potential of a general-purpose molecular assistant and suggests extensions to other scientific modalities, such as proteins, RNAs, and complexes.

Abstract

Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in task transfer, they often struggle to accurately analyze molecular features due to limited knowledge and reasoning capabilities. To address this issue, we present Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability. To this end, we design key data types that encompass the fundamental molecular features, taking into account the essential abilities for molecular reasoning. Further, to improve molecular understanding, we propose a module that integrates complementary information from different molecular encoders, leveraging the distinct advantages of molecular representations. Our experimental results demonstrate that Mol-LLaMA is capable of comprehending the general features of molecules and providing informative responses, implying its potential as a general-purpose assistant for molecular analysis. Our project page is at https://mol-llama.github.io/.

Mol-LLaMA: Towards General Understanding of Molecules in Large Molecular Language Model

TL;DR

Mol-LLaMA introduces a general-purpose molecular language model that learns broad molecular knowledge with explainability and reasoning. It couples two molecular encoders (2D MoleculeSTM and 3D UniMol) through a 2D-3D blending module and a Q-Former projector to an LLM, all trained via a two-stage process: molecular representation learning and end-to-end instruction tuning with a large, domain-specific dataset (Mol-LLaMA-Instruct) containing 284k high-quality samples. The approach yields superior performance on general molecular understanding, molecular property prediction (PAMPA/BBP), and the MoleculeQA benchmark, outperforming prior molecular LLMs and baselines while enabling diverse prompt types and few-shot capabilities. The work demonstrates the potential of a general-purpose molecular assistant and suggests extensions to other scientific modalities, such as proteins, RNAs, and complexes.

Abstract

Understanding molecules is key to understanding organisms and driving advances in drug discovery, requiring interdisciplinary knowledge across chemistry and biology. Although large molecular language models have achieved notable success in task transfer, they often struggle to accurately analyze molecular features due to limited knowledge and reasoning capabilities. To address this issue, we present Mol-LLaMA, a large molecular language model that grasps the general knowledge centered on molecules and exhibits explainability and reasoning ability. To this end, we design key data types that encompass the fundamental molecular features, taking into account the essential abilities for molecular reasoning. Further, to improve molecular understanding, we propose a module that integrates complementary information from different molecular encoders, leveraging the distinct advantages of molecular representations. Our experimental results demonstrate that Mol-LLaMA is capable of comprehending the general features of molecules and providing informative responses, implying its potential as a general-purpose assistant for molecular analysis. Our project page is at https://mol-llama.github.io/.

Paper Structure

This paper contains 65 sections, 3 figures, 39 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Molecular Foundation Models
  4. Large Language Models on Scientific Discovery
  5. Large Molecular Language Models
  6. Mol-LLaMA
  7. Instruction Dataset Construction for Comprehending Molecular Features
  8. Goal
  9. Data Generation Pipeline for Molecular Reasoning
  10. Model Architecture
  11. Molecular Encoders
  12. 2D-3D Blending Module
  13. Projector
  14. Large Language Models
  15. Training
  16. ...and 50 more sections

Figures (3)

  • Figure 1: Illustration of the end-to-end instruction tuning stage of Mol-LLaMA. It is trained on the proposed instruction datasets, where the blending module, Q-Former, and LoRA in LLMs are trained, while the molecular encoders and LLM are frozen.
  • Figure 2: Input molecule
  • Figure 3: Detail illustration of blending module and Q-Former and their training.