Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DrugLLM: Open Large Language Model for Few-shot Molecule Generation

Xianggen Liu, Yan Guo, Haoran Li, Jin Liu, Shudong Huang, Bowen Ke, Jiancheng Lv

TL;DR

DrugLLM introduces a transformer-based large language model tailored for drug design by leveraging Group-based Molecular Representation (GMR) to convert molecules into structured, token-efficient sequences. It trains on modification paragraphs that link molecular structure to properties, enabling few-shot molecule generation and optimization via autoregressive prediction of the next molecule given past modifications; the objective can be viewed as maximizing $P(x_t|x_{<t})$ over token sequences. Empirically, DrugLLM (with GMR) achieves superior few-shot performance for physicochemical properties and biological activities, and demonstrates zero-shot instruction-guided optimization that outperforms leading LLMs in several tasks. The work highlights a scalable path toward AI-assisted drug design, capable of generating novel molecules with desirable properties from limited examples, albeit with current limitations on shot count and scope of zero-shot tasks.

Abstract

Large Language Models (LLMs) have made great strides in areas such as language processing and computer vision. Despite the emergence of diverse techniques to improve few-shot learning capacity, current LLMs fall short in handling the languages in biology and chemistry. For example, they are struggling to capture the relationship between molecule structure and pharmacochemical properties. Consequently, the few-shot learning capacity of small-molecule drug modification remains impeded. In this work, we introduced DrugLLM, a LLM tailored for drug design. During the training process, we employed Group-based Molecular Representation (GMR) to represent molecules, arranging them in sequences that reflect modifications aimed at enhancing specific molecular properties. DrugLLM learns how to modify molecules in drug discovery by predicting the next molecule based on past modifications. Extensive computational experiments demonstrate that DrugLLM can generate new molecules with expected properties based on limited examples, presenting a powerful few-shot molecule generation capacity.

DrugLLM: Open Large Language Model for Few-shot Molecule Generation

TL;DR

DrugLLM introduces a transformer-based large language model tailored for drug design by leveraging Group-based Molecular Representation (GMR) to convert molecules into structured, token-efficient sequences. It trains on modification paragraphs that link molecular structure to properties, enabling few-shot molecule generation and optimization via autoregressive prediction of the next molecule given past modifications; the objective can be viewed as maximizing over token sequences. Empirically, DrugLLM (with GMR) achieves superior few-shot performance for physicochemical properties and biological activities, and demonstrates zero-shot instruction-guided optimization that outperforms leading LLMs in several tasks. The work highlights a scalable path toward AI-assisted drug design, capable of generating novel molecules with desirable properties from limited examples, albeit with current limitations on shot count and scope of zero-shot tasks.

Abstract

Large Language Models (LLMs) have made great strides in areas such as language processing and computer vision. Despite the emergence of diverse techniques to improve few-shot learning capacity, current LLMs fall short in handling the languages in biology and chemistry. For example, they are struggling to capture the relationship between molecule structure and pharmacochemical properties. Consequently, the few-shot learning capacity of small-molecule drug modification remains impeded. In this work, we introduced DrugLLM, a LLM tailored for drug design. During the training process, we employed Group-based Molecular Representation (GMR) to represent molecules, arranging them in sequences that reflect modifications aimed at enhancing specific molecular properties. DrugLLM learns how to modify molecules in drug discovery by predicting the next molecule based on past modifications. Extensive computational experiments demonstrate that DrugLLM can generate new molecules with expected properties based on limited examples, presenting a powerful few-shot molecule generation capacity.
Paper Structure (12 sections, 4 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 12 sections, 4 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Results
  3. The DrugLLM Framework
  4. DrugLLM is a few-shot learner in molecule optimization toward physicochemical properties
  5. DrugLLM is a few-shot learner in molecule optimization towards biological activities
  6. DrugLLM support instruction guided molecule optimization in a zero-shot manner
  7. Discussion
  8. METHOD DETAILS
  9. Data collection and preparation
  10. Group-based molecular representation (GMR)
  11. The implementations of DrugLLM
  12. The implementations of the competitive baselines

Figures (3)

  • Figure 1: Schematic overview of the DrugLLM framework. A, Construction process. Group-based Molecular Representation (GMR) is constructed from molecular structure units. B, Training framework. DrugLLM is trained on molecular modifications, with each paragraph representing a unique attribute. Each paragraph is self-contained and represents multiple characteristics, with different paragraphs corresponding to different attributes.
  • Figure 2: Visualization of few-shot molecule optimization. A, The training and testing examples of few-shot molecule optimization. B, Chemical space navigation by transfer learning. The UMAP plot shows the distribution of 15000 molecules selected from the source space and their corresponding 15000 molecules in the generated space after LogP property optimization. Different values of LogP are represented by different colors.
  • Figure 3: The performance of few-shot molecule optimization in physiochemical properties. A, The distribution of the water-octanol partition coefficient (LogP), Solubility, Synthetic Accessibility, and Topological Polar Surface Area (TPSA) for the source data and the generated data, represented by Kernel Density Estimation (KDE). The KDE demonstrations of the source data and the generated data are displayed by blue solid lines and red solid lines, respectively. Histograms (hist) for the source data and the generated data are represented by blue bars and red bars, respectively. B, The performance of the generation methods in terms of the success rates and the generation similarities.