Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From drift to adaptation to the failed ml model: Transfer Learning in Industrial MLOps

Waqar Muhammad Ashraf, Talha Ansar, Fahad Ahmed, Jawad Hussain, Muhammad Mujtaba Abbas, Vivek Dua

TL;DR

This work tackles drift-induced failures of production ML models in industrial settings by systematically comparing three transfer-learning update strategies—ETL, ALTL, and LLTL—applied to a feedforward ANN monitoring flue gas differential pressure in a 660 MW plant. Using two batch sizes (5 days and 8 days) and a failure window in April, the study shows ETL yields superior accuracy for small batches, whereas ALTL excels for larger batches, with SHAP indicating stable feature importance and weight-space analyses highlighting distinct plasticity-stability trade-offs. The results offer practical guidance for MLOps on selecting update strategies and batch windows to adapt failed models to data drift while managing computational costs. Overall, the paper provides mechanistic insight into how transfer-learning–driven updates behave under drift in batch-process industrial contexts and how to operationalize them in production.

Abstract

Model adaptation to production environment is critical for reliable Machine Learning Operations (MLOps), less attention is paid to developing systematic framework for updating the ML models when they fail under data drift. This paper compares the transfer learning enabled model update strategies including ensemble transfer learning (ETL), all-layers transfer learning (ALTL), and last-layer transfer learning (LLTL) for updating the failed feedforward artificial neural network (ANN) model. The flue gas differential pressure across the air preheater unit installed in a 660 MW thermal power plant is analyzed as a case study since it mimics the batch processes due to load cycling in the power plant. Updating the failed ANN model by three transfer learning techniques reveals that ETL provides relatively higher predictive accuracy for the batch size of 5 days than those of LLTL and ALTL. However, ALTL is found to be suitable for effective update of the model trained on large batch size (8 days). A mixed trend is observed for computational requirement (hyperparameter tuning and model training) of model update techniques for different batch sizes. These fundamental and empiric insights obtained from the batch process-based industrial case study can assist the MLOps practitioners in adapting the failed models to data drifts for the accurate monitoring of industrial processes.

From drift to adaptation to the failed ml model: Transfer Learning in Industrial MLOps

TL;DR

This work tackles drift-induced failures of production ML models in industrial settings by systematically comparing three transfer-learning update strategies—ETL, ALTL, and LLTL—applied to a feedforward ANN monitoring flue gas differential pressure in a 660 MW plant. Using two batch sizes (5 days and 8 days) and a failure window in April, the study shows ETL yields superior accuracy for small batches, whereas ALTL excels for larger batches, with SHAP indicating stable feature importance and weight-space analyses highlighting distinct plasticity-stability trade-offs. The results offer practical guidance for MLOps on selecting update strategies and batch windows to adapt failed models to data drift while managing computational costs. Overall, the paper provides mechanistic insight into how transfer-learning–driven updates behave under drift in batch-process industrial contexts and how to operationalize them in production.

Abstract

Model adaptation to production environment is critical for reliable Machine Learning Operations (MLOps), less attention is paid to developing systematic framework for updating the ML models when they fail under data drift. This paper compares the transfer learning enabled model update strategies including ensemble transfer learning (ETL), all-layers transfer learning (ALTL), and last-layer transfer learning (LLTL) for updating the failed feedforward artificial neural network (ANN) model. The flue gas differential pressure across the air preheater unit installed in a 660 MW thermal power plant is analyzed as a case study since it mimics the batch processes due to load cycling in the power plant. Updating the failed ANN model by three transfer learning techniques reveals that ETL provides relatively higher predictive accuracy for the batch size of 5 days than those of LLTL and ALTL. However, ALTL is found to be suitable for effective update of the model trained on large batch size (8 days). A mixed trend is observed for computational requirement (hyperparameter tuning and model training) of model update techniques for different batch sizes. These fundamental and empiric insights obtained from the batch process-based industrial case study can assist the MLOps practitioners in adapting the failed models to data drifts for the accurate monitoring of industrial processes.
Paper Structure (9 sections, 6 figures)

This paper contains 9 sections, 6 figures.

Table of Contents

  1. Introduction
  2. Method
  3. Results
  4. Data Characteristics
  5. Training of ANN model and update under two different batch sizes
  6. Insights about layer-wise weight space distribution after model update
  7. Evolution of feature importance
  8. Computational Performance of model update techniques
  9. Conclusion

Figures (6)

  • Figure 1: Schematic on transfer learning-based model update framework for process monitoring.
  • Figure 2: Characteristics of data collected for heat exchange across air preheater. (a) Distribution shift is observable in data collected for month of January and April. (b) Correlation structure between the operating variables is also changed for the two months (lower diagonal: January, upper diagonal: April).
  • Figure 3: MLOps of ANN models trained on batch sizes of (a) 5 days and (b) 8 days to monitor flue gas DP across air preheater. The trained ANN model exhibited good monitoring performance in the month of January. However, they failed during 7-10 April and model update techniques are implemented. Actual and model-based responses are also mapped corresponding to the failure zone. Peformance of models updated by LLTL, ALTL, ETL techniques on remaining April data is shown in (c,d). (e,f) parity plot to compare actual and model-based response for three model update techniques in the remaining april data.
  • Figure 4: Weight space distribution changes with model update for the batch size of (a) 5 days, and (b) 8 days. In common, last-layer weight space is widened to adapt with data-drift.
  • Figure 5: Evolution of SHAP-based feature importance with model update for ETL, LLTL and ALTL with respect to batch size of (a) 5 days and (b) 8 days. While Secondary Air Outlet Temperature (°C) is consistently the most dominant feature, but during periods of information scarcity, Primary Air Outlet Temperature (°C) exhibits a transient increase in importance to compensate for the lack of information of the new regime (V6 in (a) and V3 in (b)). Once sufficient data is accumulated, the model stabilizes, and Sec Air Outlet Temperature (°C) regains its primary significance.
  • ...and 1 more figures