NMRGym: A Comprehensive Benchmark for Nuclear Magnetic Resonance Based Molecular Structure Elucidation

TL;DR

NMRGym tackles the critical gap between synthetic and real NMR data by providing the largest standardized experimental NMR benchmark (269,999 molecules with $^1$H/$^{13}$C spectra) and a scaffold-based data split to prevent leakage. It unifies data formats, adds peak-atom annotations, and defines four downstream tasks—structure elucidation, functional-group and toxicity prediction, and spectral simulation—with an open-source leaderboard to standardize evaluation. Across baselines, transformer-based methods lead in structure elucidation, diffusion models struggle to match exact structures on real data, and 2D spectral constraints plus large pretraining improve performance; however, domain adaptation, interpretability, and 2D data scarcity remain open challenges. Overall, NMRGym provides a crucial resource and evaluation framework to accelerate robust AI-driven NMR structure elucidation and spectral prediction in real-world chemistry.

Abstract

Nuclear Magnetic Resonance (NMR) spectroscopy is the cornerstone of small-molecule structure elucidation. While deep learning has demonstrated significant potential in automating structure elucidation and spectral simulation, current progress is severely impeded by the reliance on synthetic datasets, which introduces significant domain shifts when applied to real-world experimental spectra. Furthermore, the lack of standardized evaluation protocols and rigorous data splitting strategies frequently leads to unfair comparisons and data leakage. To address these challenges, we introduce \textbf{NMRGym}, the largest and most comprehensive standardized dataset and benchmark derived from high-quality experimental NMR data to date. Comprising \textbf{269,999} unique molecules paired with high-fidelity $^1$H and $^{13}$C spectra, NMRGym bridges the critical gap between synthetic approximations and real-world diversity. We implement a strict quality control pipeline and unify data formats to ensure fair comparison. To strictly prevent data leakage, we enforce a scaffold-based split. Additionally, we provide fine-grained peak-atom level annotations to support future usage. Leveraging this resource, we establish a comprehensive evaluation suite covering diverse downstream tasks, including structure elucidation, functional group prediction from NMR, toxicity prediction from NMR, and spectral simulation, benchmarking representative state-of-the-art methodologies. Finally, we release an open-source leadboard with an automated leaderboard to foster community collaboration and standardize future research. The dataset, benchmark and leaderboard are publicly available at \textcolor{blue}{https://AIMS-Lab-HKUSTGZ.github.io/NMRGym/}.

Figures (4)

• Figure 1: Overview of the NMRGym data curation workflow. The pipeline integrates data acquisition from heterogeneous sources, rigorous quality control for standardization, and comprehensive label annotation (including toxicity, functional groups, and peak-atom assignments) to support downstream generative and predictive tasks.
• Figure 2: Comparison of data leakage between Random (red) and Scaffold (blue) splits. (a) Distribution of Maximum Tanimoto Similarity between test and training sets. The scaffold split shows a distinct shift towards lower similarity. (b) Coverage@$\tau$ curves measuring the fraction of test molecules with structural neighbors in the training set ($\text{similarity} \ge \tau$).
• Figure 3: Qualitative visualization of structural elucidation results. We categorize the resultss into three representative scenarios: (a)Accurate Elucidation. (b)High-Similarity Deviations. (c)Low-Similarity Failures. Note: "Sim" denotes the Tanimoto Similarity calculated using Morgan fingerprints.
• Figure 4: Performance comparison on the hard subset. Evaluation of Sequence Accuracy, Token Accuracy, and Morgan Tanimoto Similarity across Top-1, Top-5, and Top-10 rankings. NMRSolver (utilizing the full pipeline with the optimization module) is compared against NMRMind and the baseline ChefNMR. The results demonstrate the efficacy of the optimization strategy in refining structural predictions.