Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization

TL;DR

This paper proposes to train generative networks to minimize a predictive-sequential (or prequential) scoring rule on a recorded temporal sequence of the phenomenon of interest, which is appealing as it corresponds to the way forecasting systems are routinely evaluated.

Abstract

Probabilistic forecasting relies on past observations to provide a probability distribution for a future outcome, which is often evaluated against the realization using a scoring rule. Here, we perform probabilistic forecasting with generative neural networks, which parametrize distributions on high-dimensional spaces by transforming draws from a latent variable. Generative networks are typically trained in an adversarial framework. In contrast, we propose to train generative networks to minimize a predictive-sequential (or prequential) scoring rule on a recorded temporal sequence of the phenomenon of interest, which is appealing as it corresponds to the way forecasting systems are routinely evaluated. Adversarial-free minimization is possible for some scoring rules; hence, our framework avoids the cumbersome hyperparameter tuning and uncertainty underestimation due to unstable adversarial training, thus unlocking reliable use of generative networks in probabilistic forecasting. Further, we prove consistency of the minimizer of our objective with dependent data, while adversarial training assumes independence. We perform simulation studies on two chaotic dynamical models and a benchmark data set of global weather observations; for this last example, we define scoring rules for spatial data by drawing from the relevant literature. Our method outperforms state-of-the-art adversarial approaches, especially in probabilistic calibration, while requiring less hyperparameter tuning.

Figures (11)

• Figure 1: Estimation of the SR evaluating the forecast $P^\phi_{t+l}(\cdot|\mathbf{y}_{t-k+1:t})$ for the realization $\mathbf{y}_{t+l}$. The prequential SR is obtained by repeating this procedure for all $t$'s and summing the scores.
• Figure 2: Results for selected methods for Lorenz63 and Lorenz96 (first data component): median forecasts (solid line) and 99% credible area (shaded area) for a part of the test set. For each $t$, forecasts are obtained using the previous observation window. Credible regions for GAN and WGAN-GP are broader but contain the truth less frequently.
• Figure 3: Realization and example of predictions obtained with the patched Energy Score (patch size 16) for a specific date in the test set for the WeatherBench data set. The predictions capture the main features but are slightly different from each other.
• Figure 4: Patched SR: a SR for multivariate data is computed on localized patches, and the resulting values are summed.
• Figure 5: U-NET architecture.

Theorems & Definitions (21)

• Definition 1
• Theorem 2
• Theorem 3
• Lemma 4
• Theorem 5
• Definition 6
• Theorem 7
• Theorem 8
• Theorem 9
• Corollary 10