OCSU: Optical Chemical Structure Understanding for Molecule-centric Scientific Discovery

TL;DR

OCSU addresses the challenge of translating chemical structure diagrams into readable strings at multiple levels, enabling both chemist-friendly and machine-friendly outputs. It explores two paradigms—OCSR-based DoubleCheck and end-to-end OCSR-free Mol-VL—grounded by Vis-CheBI20, a large-scale dataset for four OCSU tasks plus functional-group recognition. Experiments show that the two approaches provide strong baseline performance, with DoubleCheck improving atom-level recognition and Mol-VL delivering strong end-to-end captioning, highlighting complementary strengths. This work advances molecule-centric discovery by enabling multi-level diagram understanding and integrating with downstream AI tools and LLMs, while releasing data and code to accelerate further research.

Abstract

Understanding the chemical structure from a graphical representation of a molecule is a challenging image caption task that would greatly benefit molecule-centric scientific discovery. Variations in molecular images and caption subtasks pose a significant challenge in both image representation learning and task modeling. Yet, existing methods only focus on a specific caption task that translates a molecular image into its graph structure, i.e., OCSR. In this paper, we propose the Optical Chemical Structure Understanding (OCSU) task, which extends low-level recognition to multilevel understanding and aims to translate chemical structure diagrams into readable strings for both machine and chemist. To facilitate the development of OCSU technology, we explore both OCSR-based and OCSR-free paradigms. We propose DoubleCheck to enhance OCSR performance via attentive feature enhancement for local ambiguous atoms. It can be cascaded with existing SMILES-based molecule understanding methods to achieve OCSU. Meanwhile, Mol-VL is a vision-language model end-to-end optimized for OCSU. We also construct Vis-CheBI20, the first large-scale OCSU dataset. Through comprehensive experiments, we demonstrate the proposed approaches excel at providing chemist-readable caption for chemical structure diagrams, which provide solid baselines for further research. Our code, model, and data are open-sourced at https://github.com/PharMolix/OCSU.

Figures (9)

• Figure 1: Introduction of OCSU. (a) Example of OCSU task. Optical chemical structure understanding is a special image caption task that describes the molecular diagrams from multiple levels, including four typical subtasks, i.e., functional group caption, molecule description, chemist-readable IUPAC naming, and machine-readable SMILES naming. (b) Two technical paradigms for OCSU. OCSR-based paradigm can fully leverage the power of existing OCSR and SMILES-based task-specific molecule understanding methods, while models are optimized in an end-to-end manner via multitask learning within OCSR-free paradigm.
• Figure 2: Exploration on OCSR-based and OCSR-free paradigms for OCSU. (a) Architecture of DoubleCheck. An attentive feature enhancement module is introduced for local ambiguous atoms. (b) Architecture of Mol-VL. A vision-language model is end-to-end optimized via multi-task learning.
• Figure 3: Performance evaluation on OCSR. (1) Performance on Vis-CheBI20. The performance advantage of DoubleCheck demonstrate the effectiveness of the proposed feature enhancement mechanism. (2) Performance on USPTO. DoubleCheck outperforms MolScribe on real-world patent scenario. (3) Performance on ACS. DoubleCheck surpasses MolScribe on real-world journal scenario.
• Figure 4: A qualitative example of Mol-VL-7B and an application example in practical scenario.
• Figure 5: Statistical analysis on functional groups. We have separately counted the distribution of functional groups in the training set and the test set. The most common functional groups in the training set and the test set are shown in (a) and (b), respectively. For the “others” category, we further counted and displayed the top 35 most frequent ones in (c) and (d).