Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Manufacturing Quality Prediction Models through the Integration of Explainability Methods

Dennis Gross, Helge Spieker, Arnaud Gotlieb, Ricardo Knoblauch

TL;DR

This work tackles milling process quality prediction under data scarcity by integrating explainability methods to prune input features. It combines decision-tree–based regressors with permutation importance and SHAP values to identify and remove less informative features, then retrains on the pruned data, achieving improved accuracy (MAPE around $4.58\%$–$4.88\%$ with GB and RF). The study demonstrates that feature pruning guided by explainability can reduce sensor needs and computation while maintaining or boosting predictive performance, and discusses differences in explanations between permutation importance and SHAP. The findings offer a practical path toward cost-efficient, interpretable predictive maintenance in manufacturing and suggest extensions to other processes and digital-twin applications.

Abstract

This research presents a method that utilizes explainability techniques to amplify the performance of machine learning (ML) models in forecasting the quality of milling processes, as demonstrated in this paper through a manufacturing use case. The methodology entails the initial training of ML models, followed by a fine-tuning phase where irrelevant features identified through explainability methods are eliminated. This procedural refinement results in performance enhancements, paving the way for potential reductions in manufacturing costs and a better understanding of the trained ML models. This study highlights the usefulness of explainability techniques in both explaining and optimizing predictive models in the manufacturing realm.

Enhancing Manufacturing Quality Prediction Models through the Integration of Explainability Methods

TL;DR

This work tackles milling process quality prediction under data scarcity by integrating explainability methods to prune input features. It combines decision-tree–based regressors with permutation importance and SHAP values to identify and remove less informative features, then retrains on the pruned data, achieving improved accuracy (MAPE around – with GB and RF). The study demonstrates that feature pruning guided by explainability can reduce sensor needs and computation while maintaining or boosting predictive performance, and discusses differences in explanations between permutation importance and SHAP. The findings offer a practical path toward cost-efficient, interpretable predictive maintenance in manufacturing and suggest extensions to other processes and digital-twin applications.

Abstract

This research presents a method that utilizes explainability techniques to amplify the performance of machine learning (ML) models in forecasting the quality of milling processes, as demonstrated in this paper through a manufacturing use case. The methodology entails the initial training of ML models, followed by a fine-tuning phase where irrelevant features identified through explainability methods are eliminated. This procedural refinement results in performance enhancements, paving the way for potential reductions in manufacturing costs and a better understanding of the trained ML models. This study highlights the usefulness of explainability techniques in both explaining and optimizing predictive models in the manufacturing realm.
Paper Structure (34 sections, 5 equations, 4 figures)

This paper contains 34 sections, 5 equations, 4 figures.

Table of Contents

  1. INTRODUCTION
  2. Lack of data.
  3. Explainability problem.
  4. Leveraging explainability techniques for optimized model training.
  5. Approach.
  6. Plan of the paper.
  7. Related Work
  8. METHODOLOGY
  9. Machine Learning Models
  10. Decision Tree Regression Models.
  11. Gradient Boosting Regression Models.
  12. Random Tree Regression Models.
  13. Explainability Methods
  14. Feature permutation importance.
  15. Shapley values.
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Milling machine that produces workpieces.
  • Figure 2: The ML prediction model receives the box plots (for time and frequency domains) and machine configuration parameters to output the quality measures.
  • Figure 3: Visualization demonstrating the feature importance rank of Rdqmaxmean predictions, as elucidated by the feature permutation importance permutation (FPI) and Shapley value (SHAP) method.
  • Figure 4: Using a different percentage of the most important features based on the different methods for the Rdq prediction. $FS$ refers to feature selection.