Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatio-temporal probabilistic forecast using MMAF-guided learning

Leonardo Bardi, Imma Valentina Curato, Lorenzo Proietti

Abstract

We employ stochastic feed-forward neural networks with Gaussian-distributed weights to determine a probabilistic forecast for spatio-temporal raster datasets. The networks are trained using MMAF-guided learning, a generalized Bayesian methodology in which the observed data are preprocessed using an embedding designed to produce a low-dimensional representation that captures their dependence and causal structure. The design of the embedding is theory-guided by the assumption that a spatio-temporal Ornstein-Uhlenbeck process with finite second-order moments generates the observed data. The trained networks, in inference mode, are then used to generate ensemble forecasts by applying different initial conditions at different horizons. Experiments conducted on both synthetic and real data demonstrate that our forecasts remain calibrated across multiple time horizons. Moreover, we show that on such data, simple feed-forward architectures can achieve performance comparable to, and in some cases better than, convolutional or diffusion deep learning architectures used in probabilistic forecasting tasks.

Spatio-temporal probabilistic forecast using MMAF-guided learning

Abstract

We employ stochastic feed-forward neural networks with Gaussian-distributed weights to determine a probabilistic forecast for spatio-temporal raster datasets. The networks are trained using MMAF-guided learning, a generalized Bayesian methodology in which the observed data are preprocessed using an embedding designed to produce a low-dimensional representation that captures their dependence and causal structure. The design of the embedding is theory-guided by the assumption that a spatio-temporal Ornstein-Uhlenbeck process with finite second-order moments generates the observed data. The trained networks, in inference mode, are then used to generate ensemble forecasts by applying different initial conditions at different horizons. Experiments conducted on both synthetic and real data demonstrate that our forecasts remain calibrated across multiple time horizons. Moreover, we show that on such data, simple feed-forward architectures can achieve performance comparable to, and in some cases better than, convolutional or diffusion deep learning architectures used in probabilistic forecasting tasks.
Paper Structure (19 sections, 1 theorem, 38 equations, 9 figures, 5 tables, 3 algorithms)

This paper contains 19 sections, 1 theorem, 38 equations, 9 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Spatio-temporal embedding and features
  3. Modeling Assumptions
  4. Features Extraction Routine
  5. MMAF-guided learning for stochastic feed-forward neural networks
  6. Optimization and Training
  7. Causal Forecast and Inference
  8. Experiments
  9. Data set and Training set-ups
  10. Evaluation Metrics
  11. Validation of the reference distribution
  12. Baseline Models
  13. Results and Discussions
  14. MMAF-guided learning applied to stochastic-feed forward neural networks
  15. Comparison of model results
  16. ...and 4 more sections

Key Result

Lemma B.1

The stochastic process $\boldsymbol{S}=(\boldsymbol{S}_i)_{i\in\mathbb{Z}}:=(\boldsymbol{X}_i,\boldsymbol{Y}_i)_{i\in\mathbb{Z}}$ related to the position $x^*$ and with values in $\mathbb{R}^{D+1}$, as defined in Section emb, is $\theta$-weakly dependent.

Figures (9)

  • Figure 1: Feature extraction with parameters $c=1$, $p=1$, and $a=3$.
  • Figure 2: The black dots indicate the spatial-temporal index grid $\mathbb{R} \times \mathbb{R}$, corresponding to an input feature $X_i=(Z_{i_1}(x_1),Z_{i_2}(x_2),Z_{i_3}(x_3)$ for $(i_s,x_s) \in I(t_0+ia,x^*)$. The pink area identifies the future cone of influence related to the spatial-time position $(i_1,x_1)$, the orange area the future cone of influence from $(i_2,x_2)$, and the yellow one the future cone of influence from $(i_3, x_3)$. All three spatio-temporal positions together influence the future realizations of the field at the spatio-temporal points in the red area, which is a subset of $\mathbb{R} \times \mathbb{R}$.
  • Figure 3: Comparison of the computational cost in FLOPs of a training iteration for an ensemble of $8$ stochastic feed-forward neural networks against the baseline models in the case of the OLR data set.
  • Figure 4: Comparison between the ensemble of stochastic feed-forward neural networks and the baseline models on the test set for the datasets GAU, NIG, and OLR with respect to the CRPS and RMSE score and the number of model parameters.
  • Figure 5: Comparison between the ensemble of stochastic feed-forward neural networks and different diffusion frameworks on the test set for the datasets GAU, NIG, and OLR with respect to the CRPS and RMSE score and the number of model parameters.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Definition 2.1
  • Definition 2.2
  • Definition 3.1
  • Lemma B.1
  • Definition B.2
  • proof