Mol-LLM: Multimodal Generalist Molecular LLM with Improved Graph Utilization

TL;DR

Mol-LLM introduces a truly multimodal generalist molecular LLM by fusing a hybrid graph encoder with a cross-modal Q-Former and a capable backbone LLM, trained via staged multimodal objectives and a novel Molecular Structure Preference Optimization (MolPO). A dedicated GNN pre-training regime emphasizes functional-group cognition and SELFIES reconstruction, and an extensive molecule-focused instruction-tuning dataset supports broad task coverage. MolPO encourages the model to distinguish aligned versus perturbed molecular graphs, improving graph utilization across diverse tasks, while Stage 1–2 pre-training stabilizes multimodal integration. Empirically, Mol-LLM achieves state-of-the-art or competitive results across property prediction, reaction forecasting, description-guided generation, captioning, and strong OOD generalization, demonstrating the value of combining graph information with multi-task instruction tuning for molecular AI applications.

Abstract

Recent advances in large language models (LLMs) have led to models that tackle diverse molecular tasks, such as chemical reaction prediction and molecular property prediction. Large-scale molecular instruction-tuning datasets have enabled sequence-only (e.g., SMILES or SELFIES) generalist molecular LLMs, and researchers are now exploring multimodal approaches that incorporate molecular structural information for further gains. However, a genuinely multimodal, generalist LLM that covers a broad spectrum of molecular tasks has yet to be fully investigated. We observe that naive next token prediction training ignores graph-structural information, limiting an LLM's ability to exploit molecular graphs. To address this, we propose (i) Molecular structure Preference Optimization (MolPO), which facilitates graph usage by optimizing preferences between pairs of correct and perturbed molecular structures, and (ii) an advanced graph encoder with a tailored pre-training strategy to improve the effect of graph utilization by MolPO. Building on these contributions, we introduce Mol-LLM, the first multimodal generalist model that (a) handles a broad spectrum of molecular tasks among molecular LLMs, (b) explicitly leverages molecular-structure information, and (c) takes advantage of extensive instruction tuning. Mol-LLM attains state-of-the-art or comparable results across the most comprehensive molecular-LLM benchmark-even on out-of-distribution datasets for reaction and property prediction, where it surpasses prior generalist molecular LLMs by a large margin.

Figures (7)

• Figure 1: (Left) Performance comparison among generalist molecular LLMs with normalized primary metrics. (Right) Graph utilization comparison between SFT and proposed multimodal training (MolPO). Score closer to 1 indicate better use of graph, approaching 0.5 indicate less utilization.
• Figure 2: (Left) Overall structure of Mol-LLM. Molecular graph is encoded into a fixed-length token sequence by a hybrid graph encoder, followed by a Q-Former that outputs query embeddings to feed LLM, with corresponding task instruction and molecular 1D sequence. (Right) Representative downstream molecular tasks.
• Figure 3: Overview of the three training stages and the loss function used at each stage. The training pipeline consists of a pre-training phase (Stages 1 and 2), followed by a fine-tuning phase (Stage 3). In Stage 1, all modules are trained independently and in parallel, whereas in Stages 2 and 3, the modules are trained in a unified architecture and loss function.
• Figure 4: (Left) Overview of the two graph pre-training tasks for the proposed hybrid graph encoder. Two distinct GNN backbones, GINE and TokenGT, are trained independently. (Right) Illustration of the MolPO training objective, which contrasts a chosen molecule with a rejected molecule.
• Figure 5: Comparison of fine-tuning performances on three tasks under different GNN architectures and initialization with (or without) pre-trained parameters. Each line is labeled as {GNN architecture}–{initialization}. Ours refers to models initialized with parameters obtained via our GNN pre-training method, whereas Scratch denotes models trained from random initialization. The x-axis denotes the number of training steps, and the y-axis shows the corresponding evaluation metric.