SubGrapher: Visual Fingerprinting of Chemical Structures

TL;DR

SubGrapher tackles visual fingerprinting of chemical structures by directly converting molecule and Markush structure images into substructure-based fingerprints, bypassing full molecular graph reconstruction. It employs two Mask-RCNN segmentation networks to detect 1,534 functional groups and 27 carbon backbone patterns, builds a substructure-graph, and encodes it into a count-based continuous fingerprint (SVMF). This one-step image-to-fingerprint approach yields superior substructure detection and molecule/Markush retrieval performance compared with state-of-the-art OCSR and fingerprinting methods across multiple benchmarks, including Markush structures, and remains robust to real-world patent imagery. The authors provide synthetic training data, model weights, and benchmarks publicly, enabling practical retrieval tasks and downstream predictive analyses from image collections.

Abstract

Automatic extraction of chemical structures from scientific literature plays a crucial role in accelerating research across fields ranging from drug discovery to materials science. Patent documents, in particular, contain molecular information in visual form, which is often inaccessible through traditional text-based searches. In this work, we introduce SubGrapher, a method for the visual fingerprinting of chemical structure images. Unlike conventional Optical Chemical Structure Recognition (OCSR) models that attempt to reconstruct full molecular graphs, SubGrapher focuses on extracting molecular fingerprints directly from chemical structure images. Using learning-based instance segmentation, SubGrapher identifies functional groups and carbon backbones, constructing a substructure-based fingerprint that enables chemical structure retrieval. Our approach is evaluated against state-of-the-art OCSR and fingerprinting methods, demonstrating superior retrieval performance and robustness across diverse molecular depictions. The dataset, models, and code are publicly available.

Figures (10)

• Figure 1: SubGrapher extracts a fingerprint from a molecular or Markush structure image in a document. Our approach identifies functional groups and carbon backbones in images. These substructures are then combined based on their connectivity to create a substructure-graph. Finally, this graph is converted to a fingerprint enabling substructure search, molecule and Markush structure retrieval, or any downstream predictive task.
• Figure 2: SubGrapher architecture. The SubGrapher model detects functional groups and carbon backbones using instance segmentation networks. The identified substructures are combined into a substructure-graph, which is then converted to a matrix fingerprint. Finally, the matrix is stored as a compressed vector. The resulting fingerprint is count-based, as each coefficient depends on the number of substructure occurrences, and continuous, since the values are real numbers.
• Figure 3: Substructure examples. Examples of functional groups and carbon backbones recognized by SubGrapher.
• Figure 4: Substructure detection qualitative evaluation. Examples of predicted functional groups are shown for images from patent documents (JPO, USPTO-10K-L, USPTO-Markush) and a scientific journal (JOC).
• Figure 5: Visual fingerprinting evaluation strategy. (A) First, a set of similar molecules are rendered into images. These images are subsequently converted into fingerprints using each of the evaluated methods. (B) Second, a molecule is converted into a fingerprint based on its SMILES. Its similarity is calculated against all fingerprints within the dataset. Finally, the correct molecule’s position is determined from the ranking of similarity scores.