Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SynRXN: An Open Benchmark and Curated Dataset for Computational Reaction Modeling

Tieu-Long Phan, Nhu-Ngoc Nguyen Song, Peter F. Stadler

TL;DR

SynRXN provides a provenance-aware benchmarking framework that modularizes CASP into five core tasks—reaction rebalancing, atom-to-atom mapping, reaction classification, reaction property prediction, and synthesis planning—paired with harmonized, versioned datasets and leakage-aware splits. It delivers deterministic build recipes, explicit data-manifest provenance, and gold-standard evaluation sets for upstream tasks, improving cross-paper comparability and robustness of CASP methods. The framework is demonstrated across extensive datasets with rigorous technical validation, establishing baselines and reproducible evaluation protocols that can scale with community contributions. By enabling fair longitudinal benchmarking and open data sharing, SynRXN lowers barriers to robust, real-world synthesis planning research and development.

Abstract

We present SynRXN, a unified benchmarking framework and open-data resource for computer-aided synthesis planning (CASP). SynRXN decomposes end-to-end synthesis planning into five task families, covering reaction rebalancing, atom-to-atom mapping, reaction classification, reaction property prediction, and synthesis route design. Curated, provenance-tracked reaction corpora are assembled from heterogeneous public sources into a harmonized representation and packaged as versioned datasets for each task family, with explicit source metadata, licence tags, and machine-readable manifests that record checksums, and row counts. For every task, SynRXN provides transparent splitting functions that generate leakage-aware train, validation, and test partitions, together with standardized evaluation workflows and metric suites tailored to classification, regression, and structured prediction settings. For sensitive benchmarking, we combine public training and validation data with held-out gold-standard test sets, and contamination-prone tasks such as reaction rebalancing and atom-to-atom mapping are distributed only as evaluation sets and are explicitly not intended for model training. Scripted build recipes enable bitwise-reproducible regeneration of all corpora across machines and over time, and the entire resource is released under permissive open licences to support reuse and extension. By removing dataset heterogeneity and packaging transparent, reusable evaluation scaffolding, SynRXN enables fair longitudinal comparison of CASP methods, supports rigorous ablations and stress tests along the full reaction-informatics pipeline, and lowers the barrier for practitioners who seek robust and comparable performance estimates for real-world synthesis planning workloads.

SynRXN: An Open Benchmark and Curated Dataset for Computational Reaction Modeling

TL;DR

SynRXN provides a provenance-aware benchmarking framework that modularizes CASP into five core tasks—reaction rebalancing, atom-to-atom mapping, reaction classification, reaction property prediction, and synthesis planning—paired with harmonized, versioned datasets and leakage-aware splits. It delivers deterministic build recipes, explicit data-manifest provenance, and gold-standard evaluation sets for upstream tasks, improving cross-paper comparability and robustness of CASP methods. The framework is demonstrated across extensive datasets with rigorous technical validation, establishing baselines and reproducible evaluation protocols that can scale with community contributions. By enabling fair longitudinal benchmarking and open data sharing, SynRXN lowers barriers to robust, real-world synthesis planning research and development.

Abstract

We present SynRXN, a unified benchmarking framework and open-data resource for computer-aided synthesis planning (CASP). SynRXN decomposes end-to-end synthesis planning into five task families, covering reaction rebalancing, atom-to-atom mapping, reaction classification, reaction property prediction, and synthesis route design. Curated, provenance-tracked reaction corpora are assembled from heterogeneous public sources into a harmonized representation and packaged as versioned datasets for each task family, with explicit source metadata, licence tags, and machine-readable manifests that record checksums, and row counts. For every task, SynRXN provides transparent splitting functions that generate leakage-aware train, validation, and test partitions, together with standardized evaluation workflows and metric suites tailored to classification, regression, and structured prediction settings. For sensitive benchmarking, we combine public training and validation data with held-out gold-standard test sets, and contamination-prone tasks such as reaction rebalancing and atom-to-atom mapping are distributed only as evaluation sets and are explicitly not intended for model training. Scripted build recipes enable bitwise-reproducible regeneration of all corpora across machines and over time, and the entire resource is released under permissive open licences to support reuse and extension. By removing dataset heterogeneity and packaging transparent, reusable evaluation scaffolding, SynRXN enables fair longitudinal comparison of CASP methods, supports rigorous ablations and stress tests along the full reaction-informatics pipeline, and lowers the barrier for practitioners who seek robust and comparable performance estimates for real-world synthesis planning workloads.
Paper Structure (17 sections, 15 equations, 3 figures, 7 tables)

This paper contains 17 sections, 15 equations, 3 figures, 7 tables.

Table of Contents

  1. Background and Summary
  2. Methods
  3. Reaction rebalancing
  4. Atom-to-atom mapping
  5. Reaction classification
  6. Reaction Property prediction
  7. Synthesis Planning
  8. Results
  9. Technical Validation
  10. Data
  11. Reaction rebalancing
  12. Atom-to-atom mapping
  13. Reaction classification
  14. Reaction property prediction
  15. Synthesis planning
  16. ...and 2 more sections

Figures (3)

  • Figure 1: The SynRXN framework. (A) Benchmark suite overview. Individual tasks include (B) reaction rebalancing, (C) atom-to-atom mapping, (D) reaction classification, (E) reaction property prediction, and (F) synthesis planning. All task modules provide curated datasets, splitting functions, and evaluation metrics.
  • Figure 2: Technical validation workflow for the SynRXN benchmark.
  • Figure 3: Overview of the SynRXN benchmark. (A) Data organization under the Data/ root, showing task-specific subdirectories and main tabular records. (B) Evaluation metrics employed for the different SynRXN tasks.