Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging

Savvas Sifnaios, George Arvanitakis, Fotios K. Konstantinidis, Georgios Tsimiklis, Angelos Amditis, Panayiotis Frangos

TL;DR

This work tackles the limitation of RGB-based material classification by leveraging hyperspectral imaging (900–1700 nm, 224 bands) combined with a lightweight pixel-wise classifier (P1CH). It introduces a dataset of HDPE, PET, PP, and PS plastics with Raman-based ground truth and semi-automated mask generation, and demonstrates that a 1D CNN operating on per-pixel spectra can achieve high accuracy and boundary precision. The approach outperforms state-of-the-art HS methods like HybridSN in pixel-wise tasks and supports near real-time inference, addressing practical needs in waste sorting, pharmaceuticals, and defense. The findings show strong potential for HS-driven material characterization, while also revealing limitations with black/dark plastics and suggesting directions for broader spectral use and incorporation of spatial information to further improve robustness and applicability.

Abstract

Recent advancements in computer vision, particularly in detection, segmentation, and classification, have significantly impacted various domains. However, these advancements are tied to RGB-based systems, which are insufficient for applications in industries like waste sorting, pharmaceuticals, and defense, where advanced object characterization beyond shape or color is necessary. Hyperspectral (HS) imaging, capturing both spectral and spatial information, addresses these limitations and offers advantages over conventional technologies such as X-ray fluorescence and Raman spectroscopy, particularly in terms of speed, cost, and safety. This study evaluates the potential of combining HS imaging with deep learning for material characterization. The research involves: i) designing an experimental setup with HS camera, conveyor, and controlled lighting; ii) generating a multi-object dataset of various plastics (HDPE, PET, PP, PS) with semi-automated mask generation and Raman spectroscopy-based labeling; and iii) developing a deep learning model trained on HS images for pixel-level material classification. The model achieved 99.94\% classification accuracy, demonstrating robustness in color, size, and shape invariance, and effectively handling material overlap. Limitations, such as challenges with black objects, are also discussed. Extending computer vision beyond RGB to HS imaging proves feasible, overcoming major limitations of traditional methods and showing strong potential for future applications.

A Deep Learning Approach for Pixel-level Material Classification via Hyperspectral Imaging

TL;DR

This work tackles the limitation of RGB-based material classification by leveraging hyperspectral imaging (900–1700 nm, 224 bands) combined with a lightweight pixel-wise classifier (P1CH). It introduces a dataset of HDPE, PET, PP, and PS plastics with Raman-based ground truth and semi-automated mask generation, and demonstrates that a 1D CNN operating on per-pixel spectra can achieve high accuracy and boundary precision. The approach outperforms state-of-the-art HS methods like HybridSN in pixel-wise tasks and supports near real-time inference, addressing practical needs in waste sorting, pharmaceuticals, and defense. The findings show strong potential for HS-driven material characterization, while also revealing limitations with black/dark plastics and suggesting directions for broader spectral use and incorporation of spatial information to further improve robustness and applicability.

Abstract

Recent advancements in computer vision, particularly in detection, segmentation, and classification, have significantly impacted various domains. However, these advancements are tied to RGB-based systems, which are insufficient for applications in industries like waste sorting, pharmaceuticals, and defense, where advanced object characterization beyond shape or color is necessary. Hyperspectral (HS) imaging, capturing both spectral and spatial information, addresses these limitations and offers advantages over conventional technologies such as X-ray fluorescence and Raman spectroscopy, particularly in terms of speed, cost, and safety. This study evaluates the potential of combining HS imaging with deep learning for material characterization. The research involves: i) designing an experimental setup with HS camera, conveyor, and controlled lighting; ii) generating a multi-object dataset of various plastics (HDPE, PET, PP, PS) with semi-automated mask generation and Raman spectroscopy-based labeling; and iii) developing a deep learning model trained on HS images for pixel-level material classification. The model achieved 99.94\% classification accuracy, demonstrating robustness in color, size, and shape invariance, and effectively handling material overlap. Limitations, such as challenges with black objects, are also discussed. Extending computer vision beyond RGB to HS imaging proves feasible, overcoming major limitations of traditional methods and showing strong potential for future applications.
Paper Structure (19 sections, 6 equations, 16 figures, 4 tables)

This paper contains 19 sections, 6 equations, 16 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hyperspectral Imaging Setup
  4. Imaging Spectroscopy
  5. Camera
  6. Conveyor
  7. Illumination
  8. Acquired Data
  9. Dataset
  10. Dataset description
  11. Acquisition Pipeline
  12. Ground truth mask generation
  13. Model Architecture & Training
  14. Architecture
  15. Data Pre-Processing
  16. ...and 4 more sections

Figures (16)

  • Figure 1: Example of RGB-based object misclassification. This image depicts pastry cakes that reassemble apples leading the RGB-based model to mistakenly classify the samples as apples.
  • Figure 2: The false colour, contrast stretched HS images used for the generation of the training set.
  • Figure 3: The Raman spectra for the 4 material classes. For the HDPE spectrum, the peaks at 1058 cm-1, 1123 cm-1, 1291 cm-1, 1437 cm-1, and the range from 2843-2876 cm-1 are characteristic of high-density polyethylene (HDPE). These peaks correspond to the vibrations of the molecular structure of HDPE, specifically indicating the various stretching and bending vibrations of the C-H bonds. Accordingly, for the rest of the spectra, it is pointed out that two characteristic peaks at 1607 cm-1 and 1721 cm-1 were detected in the top-right plot and correspond to the vibrations of the phenyl group in the polyester. Additionally, the peaks in the range of 1100-1200 cm-1 indicate the stretching vibrations of the C-O group. The peaks observed at 970 cm-1, 1034 cm-1, 1360 cm-1, 1453 cm-1, and 2946 cm-1 in the bottom-left plot are associated with the vibrations of the methyl group (CH3) in polypropylene, while the intense peak observed at 1010 cm-1, in the bottom-right plot, along with the peak at 1598 cm-1, suggest the presence of polystyrene.
  • Figure 4: The high-level architecture of the proposed Pixel-wise 1D Convolutional Hyperspectral (P1CH) Classifier .
  • Figure 5: Evolution of model's loss (top), and accuracy score (bottom) during training.
  • ...and 11 more figures

Theorems & Definitions (1)

  • Remark : Calibration on Training