Near-Infrared Hyperspectral Imaging Applications in Food Analysis -- Improving Algorithms and Methodologies

TL;DR

This thesis addresses applying near-infrared hyperspectral imaging (NIR-HSI) to food quality analysis by comparing CNN-based joint spatio-spectral modeling with traditional PLS on spectral and image data, across five hypotheses. It introduces learnable spectral preprocessing via a spectral convolution layer and fast IKPLS with fast cross-validation, plus reference to two open-source Python packages. The results show joint spatio-spectral modeling improves performance when both chemical and visual information are relevant, while PLS suffices for mean chemical content; chemical maps can be generated directly from NIR-HSI with CNNs, and bulk mean references enable subsample training with bias-corrected bulk predictions. Barley germination experiments are inconclusive due to dataset limitations but suggest NIR-HSI potential; overall, the work provides practical tools and guidelines for fast, accurate NIR-HSI analysis in food with open-source releases.

Abstract

This thesis investigates the application of near-infrared hyperspectral imaging (NIR-HSI) for food quality analysis. The investigation is conducted through four studies operating with five research hypotheses. For several analyses, the studies compare models based on convolutional neural networks (CNNs) and partial least squares (PLS). Generally, joint spatio-spectral analysis with CNNs outperforms spatial analysis with CNNs and spectral analysis with PLS when modeling parameters where chemical and physical visual information are relevant. When modeling chemical parameters with a 2-dimensional (2D) CNN, augmenting the CNN with an initial layer dedicated to performing spectral convolution enhances its predictive performance by learning a spectral preprocessing similar to that applied by domain experts. Still, PLS-based spectral modeling performs equally well for analysis of the mean content of chemical parameters in samples and is the recommended approach. Modeling the spatial distribution of chemical parameters with NIR-HSI is limited by the ability to obtain spatially resolved reference values. Therefore, a study used bulk mean references for chemical map generation of fat content in pork bellies. A PLS-based approach gave non-smooth chemical maps and pixel-wise predictions outside the range of 0-100\%. Conversely, a 2D CNN augmented with a spectral convolution layer mitigated all issues arising with PLS. The final study attempted to model barley's germinative capacity by analyzing NIR spectra, RGB images, and NIR-HSI images. However, the results were inconclusive due to the dataset's low degree of germination. Additionally, this thesis has led to the development of two open-sourced Python packages. The first facilitates fast PLS-based modeling, while the second facilitates very fast cross-validation of PLS and other classical machine learning models with a new algorithm.

Figures (11)

• Figure 1: An illustration of the 2D convolution (actually cross-correlation) operation. The kernel is placed with its center on the blue $5$. Element-wise multiplication follows, and the result is summed up and placed at the associated location (illustrated in red) in the output feature map. This kernel computes a feature equal to twice the average of the overlapped part of the input image.
• Figure 2: Scatter plots of the predicted protein content distribution for each of the 13 bulk samples in the test set. (a) PLS. (b) ResNet-18. This figure is copied from engstrom2023analyzing.
• Figure 3: Scatter plots of the means of the predicted protein content distribution for each of the 13 bulk samples in the test set. (a) PLS. (b) ResNet-18. This figure is copied from engstrom2023analyzing.
• Figure 4: Simulation of symmetric predicted subsample distributions for two bulks. The plots show the horizontal residuals from the subsample predictions to the identity. (a) The identity is the line of best fit for the subsample predictions, indicating a well-calibrated model minimizing subsample RMSE. (b) Scale ($1.56$) and bias ($-7.22$) corrected predicted values from (a) based on the best fit for the mean predictions. Thus, the identity becomes the line of best fit for the mean predictions, indicating a well-calibrated model minimizing bulk mean RMSE.
• Figure 5: Scatter plots of the bias and scale corrected means of the predicted protein content distribution for each of the 13 bulk samples in the test set. (a) PLS. (b) ResNet-18.