MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension

TL;DR

This work tackles the limited molecular understanding of large language models by introducing MolX, a multi-modal external module that fuses SMILES, 2D molecular graphs, and a handcrafted Morgan fingerprint into a unified embedding aligned with the LLM’s textual space. A cross-space pre-training regime, conducted with a frozen base LLM, enables MolX to produce informative embeddings for molecule-related tasks such as molecule-to-text translation and molecular property prediction, achieving superior performance with only 0.53%–0.82% of trainable parameters. The approach demonstrates strong task generalization, including unseen properties, and shows compatibility with different base LLMs, underscoring MolX’s potential as a plug-in enhancement for molecular reasoning in diverse languages models. Overall, MolX advances practical molecular understanding in LLMs while preserving their general-purpose capabilities, offering a scalable path for chemistry-aware AI systems.

Abstract

Large Language Models (LLMs) with their strong task-handling capabilities have shown remarkable advancements across a spectrum of fields, moving beyond natural language understanding. However, their proficiency within the chemistry domain remains restricted, especially in solving molecule-related tasks. This challenge is attributed to their inherent limitations in comprehending molecules using only common textual representations, i.e. SMILES strings. In this study, we seek to enhance the ability of LLMs to comprehend molecules by equipping them with a multi-modal external module, termed MolX. Instead of directly using SMILES strings to represent a molecule, we utilize specific encoders to extract fine-grained features from both SMILES string and 2D molecular graph representations for feeding into an LLM. A hand-crafted molecular fingerprint is incorporated to leverage its embedded domain knowledge. To establish an alignment between MolX and the LLM's textual input space, the model in which the LLM is frozen, is pre-trained with a strategy including a diverse set of tasks. Experimental evaluations show that our proposed method outperforms baselines across downstream molecule-related tasks ranging from molecule-to-text translation to molecular property prediction, with and without fine-tuning the LLM, while only introducing a small number of trainable parameters--0.53\% and 0.82\%, respectively.

Figures (6)

• Figure 1: Current paradigm of using an LLM for molecule-related tasks and its issues.
• Figure 2: An overview of our proposed method with the main pre-training task.
• Figure 3: Examples of auxiliary tasks in our instruction-based pre-training strategy.
• Figure 4: An example of molecular property prediction.
• Figure 5: Added results for molecule description generation.