Evolving Multi-Scale Normalization for Time Series Forecasting under Distribution Shifts

TL;DR

The effectiveness of EvoMSN is evaluated in improving the performance of five mainstream forecasting methods on benchmark datasets and also shows its superiority compared to existing advanced normalization and online learning approaches.

Abstract

Complex distribution shifts are the main obstacle to achieving accurate long-term time series forecasting. Several efforts have been conducted to capture the distribution characteristics and propose adaptive normalization techniques to alleviate the influence of distribution shifts. However, these methods neglect the intricate distribution dynamics observed from various scales and the evolving functions of distribution dynamics and normalized mapping relationships. To this end, we propose a novel model-agnostic Evolving Multi-Scale Normalization (EvoMSN) framework to tackle the distribution shift problem. Flexible normalization and denormalization are proposed based on the multi-scale statistics prediction module and adaptive ensembling. An evolving optimization strategy is designed to update the forecasting model and statistics prediction module collaboratively to track the shifting distributions. We evaluate the effectiveness of EvoMSN in improving the performance of five mainstream forecasting methods on benchmark datasets and also show its superiority compared to existing advanced normalization and online learning approaches. The code is publicly available at https://github.com/qindalin/EvoMSN.

Figures (7)

• Figure 1: The marginal distribution $P(Y)$ of a time series viewed from different scales will show diverse dynamics, while the conditional distribution $P(Y|X)$ also evolves across time.
• Figure 2: The proposed evolving multi-scale normalization framework. The overall structure includes periodicity extraction, multi-scale distribution dynamics modeling, and normalization-denormalization with adaptive ensembling. The learning strategy is designed with offline two-stage pretraining and online alternate updating to facilitate models to capture the evolving distribution.
• Figure 3: Visualization of online long-term forecasting results with the output window length of 336.
• Figure 4: Evolution of the cumulative average MSE loss during online forecasting.
• Figure 5: Visualization of statistics of windows. The blue line represents the mean of each window. Red dots represent the window mean plus/minus the window standard deviation, and the gray dash line represents the deviation range of the window. (a), (b), (c) plots the statistics of each window when the window length equals to 96, 48, and 24, respectively.