Generative Multi-Objective Bayesian Optimization with Scalable Batch Evaluations for Sample-Efficient De Novo Molecular Design

TL;DR

The paper tackles the challenge of sample-efficient, multi-objective de novo molecular design in vast chemical spaces by proposing a modular generate-then-optimize framework. It introduces qPMHI, a scalable batch acquisition that decomposes the hypervolume-improvement objective across candidates, enabling exact batch selection from large generated pools. Through logP–TPSA benchmarks and a quinone-based OEM design case, the method consistently expands the Pareto front more rapidly than state-of-the-art baselines and demonstrates strong surrogate performance with attention-augmented BGNNs. The work highlights the practical value of decoupling candidate generation from optimization, enabling flexible, generator-agnostic pipelines suitable for parallel evaluations and real-world discovery campaigns.

Abstract

Designing molecules that must satisfy multiple, often conflicting objectives is a central challenge in molecular discovery. The enormous size of chemical space and the cost of high-fidelity simulations have driven the development of machine learning-guided strategies for accelerating design with limited data. Among these, Bayesian optimization (BO) offers a principled framework for sample-efficient search, while generative models provide a mechanism to propose novel, diverse candidates beyond fixed libraries. However, existing methods that couple the two often rely on continuous latent spaces, which introduces both architectural entanglement and scalability challenges. This work introduces an alternative, modular "generate-then-optimize" framework for de novo multi-objective molecular design/discovery. At each iteration, a generative model is used to construct a large, diverse pool of candidate molecules, after which a novel acquisition function, qPMHI (multi-point Probability of Maximum Hypervolume Improvement), is used to optimally select a batch of candidates most likely to induce the largest Pareto front expansion. The key insight is that qPMHI decomposes additively, enabling exact, scalable batch selection via only simple ranking of probabilities that can be easily estimated with Monte Carlo sampling. We benchmark the framework against state-of-the-art latent-space and discrete molecular optimization methods, demonstrating significant improvements across synthetic benchmarks and application-driven tasks. Specifically, in a case study related to sustainable energy storage, we show that our approach quickly uncovers novel, diverse, and high-performing organic (quinone-based) cathode materials for aqueous redox flow battery applications.

Figures (8)

• Figure 1: Overview of the proposed two-stage de novo multi-objective molecular optimization framework. In Stage 1, a generative approach proposes a large pool of candidate molecules, which may be represented in diverse formats (e.g., graphs, SMILES, fingerprints). These candidates can be tailored to reflect user-defined biases toward specific regions of chemical space, high predicted property values, or high epistemic uncertainty. In Stage 2, a probabilistic surrogate model (trained on labeled molecules) is used to predict objective values and uncertainty. Monte Carlo posterior sampling is used to estimate the probability that each candidate achieves the maximum hypervolume improvement (HVI), yielding a simple acquisition function, qPMHI, that decomposes additively across candidates. Thus, the optimal batch can be easily selected by sorting the candidates by their maximum HVI score.
• Figure 2: Pareto front hypervolume progression over 20 iterations on the logP–TPSA benchmark. Each curve shows the average hypervolume achieved by a method (batch size $q = 50$), with shaded bands denoting 95% confidence intervals. Our method (green) achieves the highest hypervolume at each step.
• Figure 3: Final Pareto fronts for each method on the logP–TPSA benchmark (best trial per method). Each point is a non-dominated molecule sampled during optimization. Our method (green) identifies a broader frontier, particularly at high-logP / low-TPSA trade-offs.
• Figure 4: Candidate batches selected by our method (GA/BGNN) at iterations 5 (purple squares), 10 (green triangles), and 20 (orange diamonds), overlaid on a hexbin density of the initial training data. Our method progressively expands outward from the training distribution while maintaining coverage of diverse trade-off regions.
• Figure 5: Comparison of four acquisition functions (qPMHI, qEHVI, qPOTS, and Sobol) on the logP-TPSA benchmark using a fixed candidate pool. Subplots (a)--(e) show hypervolume over iterations for five replicates (batch size $q = 100$). Subplot (f) reports the mean fraction of true Pareto-optimal candidates recovered at each iteration. qPMHI consistently dominates in both metrics.