Benchmarking Deep Learning Models for Raman Spectroscopy Across Open-Source Datasets

TL;DR

This work tackles the lack of fair, cross-paper benchmarking for Raman spectroscopy classification by evaluating five Raman-specific deep learning architectures (CNN- and transformer-based) under a unified training/evaluation protocol across three open-source datasets with varied real-world challenges. It demonstrates that SANet often yields the strongest performance, but all models experience substantial degradation under distribution shift, such as dusty test conditions, highlighting brittleness to acquisition variability. The study also reveals task- and dataset-specific dynamics, with high validation accuracies not always translating to test performance, and shows near-ceiling results in the API dataset when synonyms are harmonized. Overall, the paper provides a transparent, reproducible baseline for Raman spectra classifiers and motivates the development of Raman foundation models and broader, open benchmarks to mitigate domain shifts in practical deployments.

Abstract

Deep learning classifiers for Raman spectroscopy are increasingly reported to outperform classical chemometric approaches. However their evaluations are often conducted in isolation or compared against traditional machine learning methods or trivially adapted vision-based architectures that were not originally proposed for Raman spectroscopy. As a result, direct comparisons between existing deep learning models developed specifically for Raman spectral analysis on shared open-source datasets remain scarce. To the best of our knowledge, this study presents one of the first systematic benchmarks comparing three or more published Raman-specific deep learning classifiers across multiple open-source Raman datasets. We evaluate five representative deep learning architectures under a unified training and hyperparameter tuning protocol across three open-source Raman datasets selected to support standard evaluation, fine-tuning, and explicit distribution-shift testing. We report classification accuracies and macro-averaged F1 scores to provide a fair and reproducible comparison of deep learning models for Raman spectra based classification.

Figures (3)

• Figure 1: Dataset overview and qualitative comparison across domains. Top row: the three Raman spectroscopy datasets used in this study: 1. MLROD (mineral spectra), 2. Bacteria-ID (spectra of bacterial species/strains), and 3. API dataset (spectra of active pharmaceutical ingredients). Middle row: dataset-level statistics summarizing scale and label structure including total number of classes. Bottom row: representative spectra randomly sampled from each dataset (intensity vs. Raman shift ($\text{cm}^{-1}$)).
• Figure 2: MLROD spectral shift across evaluation conditions (“clean” vs. “dusty”). Representative Raman spectra for three minerals (Biotite, Albite, Hornblende) shown across the training set (left), 'clean' test set (middle), and 'dusty test' set (right). Each row corresponds to the same material label across splits; each trace is a single example spectrum plotted as Raman intensity (arb. units) versus wavenumber ($cm^{-1}$). While the training and 'clean' test spectra largely preserve characteristic peak locations and relative band structure, the 'dusty' split exhibits pronounced contamination artifacts such as elevated and drifting baselines, broadened features, and spurious high-intensity components. This illustrates a significant distribution shift that challenges model generalization.
• Figure 3: API dataset label synonymy. Overlaid Raman spectra (five randomly selected measurements per panel) for 4-methyl-2-pentanone (left) and methyl isobutyl ketone (right), plotted as Raman intensity (arb. units) versus wavenumber ($cm^{-1}$). The near-identical signatures across the two panels reflect that methyl isobutyl ketone and 4-methyl-2-pentanone are chemically identical, illustrating potential label aliasing in the dataset and motivating label harmonization during our evaluation.