Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Factors with Gaussian Processes for Forecasting

Danielle C. Maddix, Yuyang Wang, Alex Smola

TL;DR

The paper tackles forecasting large collections of related time series by combining a scalable global deep-factor backbone with a local probabilistic Gaussian Process for per-series fluctuations, enabling uncertainty propagation. A global RNN generates latent factors that are loaded per series via attention, while a per-series GP captures idiosyncratic dynamics, yielding a tractable marginal likelihood for learning. The approach outperforms state-of-the-art methods like DeepAR and Prophet, especially in heavy-tail uncertainty assessment (P90 quantile loss) on electricity and traffic datasets. This global-local framework offers a principled way to leverage cross-series information while preserving per-series uncertainty, with potential applicability to large-scale forecasting tasks requiring reliable uncertainty quantification.

Abstract

A large collection of time series poses significant challenges for classical and neural forecasting approaches. Classical time series models fail to fit data well and to scale to large problems, but succeed at providing uncertainty estimates. The converse is true for deep neural networks. In this paper, we propose a hybrid model that incorporates the benefits of both approaches. Our new method is data-driven and scalable via a latent, global, deep component. It also handles uncertainty through a local classical Gaussian Process model. Our experiments demonstrate that our method obtains higher accuracy than state-of-the-art methods.

Deep Factors with Gaussian Processes for Forecasting

TL;DR

The paper tackles forecasting large collections of related time series by combining a scalable global deep-factor backbone with a local probabilistic Gaussian Process for per-series fluctuations, enabling uncertainty propagation. A global RNN generates latent factors that are loaded per series via attention, while a per-series GP captures idiosyncratic dynamics, yielding a tractable marginal likelihood for learning. The approach outperforms state-of-the-art methods like DeepAR and Prophet, especially in heavy-tail uncertainty assessment (P90 quantile loss) on electricity and traffic datasets. This global-local framework offers a principled way to leverage cross-series information while preserving per-series uncertainty, with potential applicability to large-scale forecasting tasks requiring reliable uncertainty quantification.

Abstract

A large collection of time series poses significant challenges for classical and neural forecasting approaches. Classical time series models fail to fit data well and to scale to large problems, but succeed at providing uncertainty estimates. The converse is true for deep neural networks. In this paper, we propose a hybrid model that incorporates the benefits of both approaches. Our new method is data-driven and scalable via a latent, global, deep component. It also handles uncertainty through a local classical Gaussian Process model. Our experiments demonstrate that our method obtains higher accuracy than state-of-the-art methods.

Paper Structure

This paper contains 6 sections, 4 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Deep Factor Model with Gaussian Processes
  3. Generative Model
  4. Inference and Learning
  5. Experiments
  6. Conclusion

Figures (2)

  • Figure 1: Plate graph of the proposed Deep Factors with Gaussian Processes model. The diamond nodes represent deterministic states.
  • Figure 2: The dashed orange curve shows the forecast of the proposed global LSTM with GP local model. The black vertical line marks the division between the training and prediction regions.