Models Matter: The Impact of Single-Step Retrosynthesis on Synthesis Planning

TL;DR

This work investigates how single-step retrosynthesis models influence multi-step synthesis planning by integrating state-of-the-art single-step methods into a common planning framework. It reveals a lack of direct transfer between single-step top-n performance and multi-step route finding, showing that larger, more diverse datasets alter model rankings and planning outcomes. The study demonstrates that switching single-step models can boost multi-step planning success by up to 28% and that each model yields distinct synthesis routes, highlighting the importance of planning-aware evaluation and data-driven model selection. The findings advocate for larger, more diverse benchmarks and the integration of chemical feasibility considerations to improve the practical utility of AI-driven synthesis planning.

Abstract

Retrosynthesis consists of breaking down a chemical compound recursively step-by-step into molecular precursors until a set of commercially available molecules is found with the goal to provide a synthesis route. Its two primary research directions, single-step retrosynthesis prediction, which models the chemical reaction logic, and multi-step synthesis planning, which tries to find the correct sequence of reactions, are inherently intertwined. Still, this connection is not reflected in contemporary research. In this work, we combine these two major research directions by applying multiple single-step retrosynthesis models within multi-step synthesis planning and analyzing their impact using public and proprietary reaction data. We find a disconnection between high single-step performance and potential route-finding success, suggesting that single-step models must be evaluated within synthesis planning in the future. Furthermore, we show that the commonly used single-step retrosynthesis benchmark dataset USPTO-50k is insufficient as this evaluation task does not represent model performance and scalability on larger and more diverse datasets. For multi-step synthesis planning, we show that the choice of the single-step model can improve the overall success rate of synthesis planning by up to +28% compared to the commonly used baseline model. Finally, we show that each single-step model finds unique synthesis routes, and differs in aspects such as route-finding success, the number of found synthesis routes, and chemical validity, making the combination of single-step retrosynthesis prediction and multi-step synthesis planning a crucial aspect when developing future methods.

Figures (12)

• Figure 1: Evaluation Framework for single-step models (AiZynthFinder (AZF), LocalRetro, Chemformer, and MHNreact), trained on different public (USPTO-50k, USPTO-PaRoutes-1M) and proprietary (AZ-1M, AZ-18M) datasets in synthesis planning on Caspyrus10k and PaRoutes.
• Figure 2: Single-step Retrosynthesis Prediction Performance in terms of top-n accuracy for AZF, LocalRetro, Chemformer, and MHNreact on different datasets (USPTO-50k, USPTO-PaRoutes-1M, AZ-1M, AZ-18M) (see Supplementary Table \ref{['supptab:single_step_performance']}).
• Figure 3: Multi-step synthesis planning accuracy on PaRoutes gold-standard synthesis routes with different single-step models trained on USPTO-PaRoutes-1M. Route accuracy measures the ability to recover the correct synthesis route within top-n, whereas building block accuracy measures the ability to recover the correct building blocks while not considering reactions and intermediates (see Supplementary Table \ref{['supptab:paroutes_performance']}).
• Figure 4: Caspyrus10k route statistics of top-5 found synthesis routes by different single-step retrosynthesis models trained on USPTO-PaRoutes-1M. Shown are the maximum depth, referring to the longest linear path within the route, the number of building blocks within the route, and the number of reactants per route reaction.
• Figure 5: Distribution and overlap of route clusters per single-step model and dataset when clustering with route-distance package genhedenClusteringSyntheticRoutes2021genhedenFastPredictionDistances2022. Clusters were calculated on a per molecule basis, N clusters shows the number of clusters which contained the stated combination of models.