From In Silico to In Vitro: Evaluating Molecule Generative Models for Hit Generation

TL;DR

This study reframes hit generation as a standalone task for molecule-generative models and introduces a multi-stage evaluation pipeline combining medicinal-chemistry filters, distributional/structural metrics, docking, and prospective in vitro validation. By benchmarking autoregressive models (MolRNN, GraphINVENT) and a diffusion-based model (DiGress) across general, hit-like, and target-specific datasets, it demonstrates that well-tuned autoregressive models can produce valid, diverse, and biologically relevant hit candidates, including a GSK-3β inhibitor with sub-micromolar potency validated in vitro. The work also highlights limitations of current evaluation metrics and the data bottlenecks for target-specific hit generation, particularly for diffusion-based approaches. Overall, the results suggest a practical, modular AI-driven pipeline: generate high-quality hit candidates with trained models and then apply ML-assisted or medicinal-chemistry refinement to support early-stage drug discovery, while calling for richer, curated hit-like datasets and bioactivity-centric benchmarks.

Abstract

Hit identification is a critical yet resource-intensive step in the drug discovery pipeline, traditionally relying on high-throughput screening of large compound libraries. Despite advancements in virtual screening, these methods remain time-consuming and costly. Recent progress in deep learning has enabled the development of generative models capable of learning complex molecular representations and generating novel compounds de novo. However, using ML to replace the entire drug-discovery pipeline is highly challenging. In this work, we rather investigate whether generative models can replace one step of the pipeline: hit-like molecule generation. To the best of our knowledge, this is the first study to explicitly frame hit-like molecule generation as a standalone task and empirically test whether generative models can directly support this stage of the drug discovery pipeline. Specifically, we investigate if such models can be trained to generate hit-like molecules, enabling direct incorporation into, or even substitution of, traditional hit identification workflows. We propose an evaluation framework tailored to this task, integrating physicochemical, structural, and bioactivity-related criteria within a multi-stage filtering pipeline that defines the hit-like chemical space. Two autoregressive and one diffusion-based generative models were benchmarked across various datasets and training settings, with outputs assessed using standard metrics and target-specific docking scores. Our results show that these models can generate valid, diverse, and biologically relevant compounds across multiple targets, with a few selected GSK-3$β$ hits synthesized and confirmed active in vitro. We also identify key limitations in current evaluation metrics and available training data.

Figures (4)

• Figure 1: Integrated Training and Evaluation Framework.Left: Dataset construction and model training pipeline, showing the filtering strategies applied to ChEMBL to obtain general-purpose, hit-like, and target-specific datasets, their use in training MolRNN, GraphINVENT, and DiGress models, and subsequent fine-tuning in hit-like and target-specific settings. Right: Evaluation pipeline for generated molecules, outlining the multi-stage workflow from VUN and hit-like filtering through distributional, structural, and docking-based metrics, visual inspection, and final selection of top GSK-3$\beta$ candidates for synthesis and in vitro testing.
• Figure 2: (A) Structures of selected compounds (1--3), their most similar known GSK-3$\beta$ inhibitors (4--6), and closest kinase inhibitors (7--9) by TD. (B) Dose--response curve for GSK-3$\beta$ inhibition by compound 1, shown as mean $\pm$ SD of three replicates, representative of three experiments. (C) t-SNE projection of compound 1 (red star) within the chemical space of known GSK-3$\beta$ ligands (gold dots). (D) Comparison of pChEMBL value of compound 1 with known inhibitors, shown as a violin plot (all inhibitors, light blue) and subset with hit-like properties (blue dots).
• Figure 3: Predicted binding conformations of compounds 1 (carbon atoms in gold, A), 2 (carbon atoms in purple, B), and 3 (carbon atoms in yellow, C) at the binding site of GSK-3$\beta$. In panels A--C, the protein structure is shown as a thin grey ribbon. Residues interacting with the docked compound are displayed in stick representation with light grey carbons and explicitly labelled. Hydrogen bonds are depicted as yellow dashed lines, while a grey mesh highlights the boundaries of the binding pocket within 5 Å of each ligand. The chemical structures of compounds 1, 2, and 3 are shown in panels D--F, respectively.
• Figure 4: Comparison of MW (upper panel) and logP (lower panel) of the generated molecules with respect to each target ligand set.