Recency-Weighted Temporally-Segmented Ensemble for Time-Series Modeling

TL;DR

The Recency-Weighted Temporally-Segmented ensemble model, a novel chunk-based approach for multi-step forecasting, shows promise in developing automatic, adaptable forecasting models for decision-making and control systems in process industries and other complex systems.

Abstract

Time-series modeling in process industries faces the challenge of dealing with complex, multi-faceted, and evolving data characteristics. Conventional single model approaches often struggle to capture the interplay of diverse dynamics, resulting in suboptimal forecasts. Addressing this, we introduce the Recency-Weighted Temporally-Segmented (ReWTS, pronounced `roots') ensemble model, a novel chunk-based approach for multi-step forecasting. The key characteristics of the ReWTS model are twofold: 1) It facilitates specialization of models into different dynamics by segmenting the training data into `chunks' of data and training one model per chunk. 2) During inference, an optimization procedure assesses each model on the recent past and selects the active models, such that the appropriate mixture of previously learned dynamics can be recalled to forecast the future. This method not only captures the nuances of each period, but also adapts more effectively to changes over time compared to conventional `global' models trained on all data in one go. We present a comparative analysis, utilizing two years of data from a wastewater treatment plant and a drinking water treatment plant in Norway, demonstrating the ReWTS ensemble's superiority. It consistently outperforms the global model in terms of mean squared forecasting error across various model architectures by 10-70\% on both datasets, notably exhibiting greater resilience to outliers. This approach shows promise in developing automatic, adaptable forecasting models for decision-making and control systems in process industries and other complex systems.

Figures (12)

• Figure 1: Conceptual illustration of the look-back, chunk-based approach. The figure shows how the models trained on disjoint chunks of times series are used to provide forecasts of the target response at time $t$ given look-back data.
• Figure 2: A small time segment of the first six days in January 2022 showing how the nitrate concentration in the wastewater plant varies in time. Time resolution is in 10 minutes.
• Figure 3: Figure to the left: A small time segment of the first six days in January 2021 showing how the turbidity at the drinking water plant varies in time. The peaks in turbidity indicates when the filter is flushed. Drinking water is produced between the peaks. Time resolution is 10 minutes. Figure to the right: Measurements for the same time segment, but zoomed in to see the dynamics of the turbidity during drinking water production..
• Figure 4: Data (blue) and model predictions (colors) for all chunks for the training set and the test data from the sine experiment. Note how the global model has prioritized learning the dynamics of the dominant amplitude in chunk #3 of the training set.
• Figure 5: With rapid change in the dynamics, the ReWTS ensemble model is slow to adapt (the true dynamics given by the blue line).