On the Role of Reversible Instance Normalization

Abstract

Data normalization is a crucial component of deep learning models, yet its role in time series forecasting remains insufficiently understood. In this paper, we identify three central challenges for normalization in time series forecasting: temporal input distribution shift, spatial input distribution shift, and conditional output distribution shift. In this context, we revisit the widely used Reversible Instance Normalization (RevIN), by showing through ablation studies that several of its components are redundant or even detrimental. Based on these observations, we draw new perspectives to improve RevIN's robustness and generalization.

Figures (22)

• Figure 1: Illustration of the three distribution shifts: (a) temporal shift between training and test periods (rolling average of a Traffic sensor), (b) spatial shift between users (two Solar sensors), (c) conditional shift (different horizons for similar look-back windows, from one Electricity user).
• Figure 2: Illustration of the RevIN process on three synthetic examples.
• Figure 3: Values of $\delta$ for a given user in time. Modulation stationarity does not hold.
• Figure 4: Distribution of sampled $(\delta,\lambda)$. In red from a single user, in blue from the whole set of users. Modulation stationarity does not hold.
• Figure 5: Example of a signal with saturation. A model with instance normalization cannot distinguish both blue windows nor both green windows. Yet the expected outputs are different for each.