What is to be gained by ensemble models in analysis of spectroscopic data?

TL;DR

This study tackles the question of whether ensemble methods can meaningfully improve prediction from mid-infrared spectroscopy data in milk analytics, where no single model consistently dominates across tasks. It conducts an empirical evaluation using two MIR benchmarks (regression and classification) built from milk spectra, comparing a broad library of candidate models and several stacking meta-learners whose predictions are fused via cross-validated, out-of-fold training. Linear mixed models are used to assess performance across 50 random splits (regression) and 10 splits (classification), revealing that stacking ensembles—particularly with non-negative coefficient constraints on the meta-learner—consistently outperform the best individual models, with $RMSE$ reductions and $ACC$ gains (e.g., $ACC$ rising from $0.78$ to $0.81$ in classification). The findings support using diverse ensemble strategies for spectroscopic calibration tasks, while highlighting the role of model diversity and careful cross-validation to avoid bias, and noting that linear models like PLS, LASSO, and Elastic Net remain strong competitors. Overall, ensemble stacking offers a principled path to robust predictions in chemometrics and MIR spectroscopy applications.

Abstract

An empirical study was carried out to compare different implementations of ensemble models aimed at improving prediction in spectroscopic data. A wide range of candidate models were fitted to benchmark datasets from regression and classification settings. A statistical analysis using linear mixed model was carried out on prediction performance criteria resulting from model fits over random splits of the data. The results showed that the ensemble classifiers were able to consistently outperform candidate models in our application

Figures (8)

• Figure 1: Distributions of the fourteen traits in the regression challenge dataset.
• Figure 2: Scatter plot of the MIR spectra from the classification challenge dataset projected on the space spanned by the two latent variables associated with the discriminant functions for milk from cows fed grass (GRS), clover (CLV) and nutional mix (TMR).
• Figure 3: Subsets of the of MIR spectra from the regression challenge dataset (left) and classification challenge dataset (right). In the supervised classification task (right), the colours correspond to the diet regimen of the animal whose sample is plotted.
• Figure 4: Regression dataset: LME estimates of mean RMSE with 95% CI. Overlapping intervals indicate where the difference is not statistically significant. The interaction plot shows that for most traits (x-axis), the stacking ensemble with non-negative constraint on the coefficients (Ens_nonneg, solid green line), gives lowest mean RMSE.
• Figure 5: Regression dataset: LME model estimates of mean RMSE over all random splits and traits with 95% CIs. Overlapping intervals indicate where the difference is not statistically significant. The plot shows that the stacking ensemble with the non-negative coefficients gets the lowest average RMSE over all random splits and traits. Second best is stacking ensemble with lasso (Ens_LASSO) and the top candidate algorithms are PLS, LASSO and EN. These candidate models outperform the model averaging ensemble (Ens_MA).