Towards Robust Ferrous Scrap Material Classification with Deep Learning and Conformal Prediction

TL;DR

This work tackles robust ferrous scrap material classification in steel production by quantifying prediction uncertainty through Split Conformal Prediction integrated with state-of-the-art vision models (ResNet-50, ViT, Swin) and enhanced explainability via XAI methods. On a dataset of 8,147 images across nine classes, Swin delivers the strongest reliability with average accuracy above approximately 95.5 percent and the smallest conformal prediction sets, while uncertainty quantification and heatmap-based explanations support safer industrial deployment. The combined approach enables calibrated uncertainty, improved transparency (notably with Score-CAM), and actionable insights for operators, advancing practical adoption of automated scrap sorting. Limitations include labeling ambiguity and dataset size, suggesting avenues for dataset expansion and real-world deployment studies.

Abstract

In the steel production domain, recycling ferrous scrap is essential for environmental and economic sustainability, as it reduces both energy consumption and greenhouse gas emissions. However, the classification of scrap materials poses a significant challenge, requiring advancements in automation technology. Additionally, building trust among human operators is a major obstacle. Traditional approaches often fail to quantify uncertainty and lack clarity in model decision-making, which complicates acceptance. In this article, we describe how conformal prediction can be employed to quantify uncertainty and add robustness in scrap classification. We have adapted the Split Conformal Prediction technique to seamlessly integrate with state-of-the-art computer vision models, such as the Vision Transformer (ViT), Swin Transformer, and ResNet-50, while also incorporating Explainable Artificial Intelligence (XAI) methods. We evaluate the approach using a comprehensive dataset of 8147 images spanning nine ferrous scrap classes. The application of the Split Conformal Prediction method allowed for the quantification of each model's uncertainties, which enhanced the understanding of predictions and increased the reliability of the results. Specifically, the Swin Transformer model demonstrated more reliable outcomes than the others, as evidenced by its smaller average size of prediction sets and achieving an average classification accuracy exceeding 95%. Furthermore, the Score-CAM method proved highly effective in clarifying visual features, significantly enhancing the explainability of the classification decisions.

Figures (32)

• Figure 1: Scheme for automatic image capture with a fixed camera.
• Figure 2: Image example of each class.
• Figure 3: Confusion Matrix of Resnet-50, ViT and Swin.
• Figure 4: Calibration Conformal Prediction Threshold for Resnet-50, ViT and Swin.
• Figure 5: Input Image for explainability methods comparison.