Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multivariate Online Linear Regression for Hierarchical Forecasting

Massil Hihat, Guillaume Garrigos, Adeline Fermanian, Simon Bussy

TL;DR

MultiVAW is introduced, a method that extends the well-known Vovk-Azoury-Warmuth algorithm to the multivariate setting, and it is shown that it also enjoys logarithmic regret in time.

Abstract

In this paper, we consider a deterministic online linear regression model where we allow the responses to be multivariate. To address this problem, we introduce MultiVAW, a method that extends the well-known Vovk-Azoury-Warmuth algorithm to the multivariate setting, and show that it also enjoys logarithmic regret in time. We apply our results to the online hierarchical forecasting problem and recover an algorithm from this literature as a special case, allowing us to relax the hypotheses usually made for its analysis.

Multivariate Online Linear Regression for Hierarchical Forecasting

TL;DR

MultiVAW is introduced, a method that extends the well-known Vovk-Azoury-Warmuth algorithm to the multivariate setting, and it is shown that it also enjoys logarithmic regret in time.

Abstract

In this paper, we consider a deterministic online linear regression model where we allow the responses to be multivariate. To address this problem, we introduce MultiVAW, a method that extends the well-known Vovk-Azoury-Warmuth algorithm to the multivariate setting, and show that it also enjoys logarithmic regret in time. We apply our results to the online hierarchical forecasting problem and recover an algorithm from this literature as a special case, allowing us to relax the hypotheses usually made for its analysis.
Paper Structure (31 sections, 9 theorems, 40 equations, 3 figures, 1 algorithm)

This paper contains 31 sections, 9 theorems, 40 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notations.
  3. Problem Statement
  4. Multivariate Online Linear Regression
  5. Existing Algorithms and Results
  6. The MultiVAW Algorithm
  7. Definition of MultiVAW
  8. Implementation and Complexity
  9. Regret Bounds for MultiVAW
  10. Application: Online Hierarchical Forecasting
  11. Introduction to Hierarchical Forecasting
  12. Online Hierarchical Forecasting
  13. Related Work and MetaVAW
  14. MultiVAW for Online Hierarchical Forecasting
  15. Regret Guarantees for MultiVAW-OHF
  16. ...and 16 more sections

Key Result

Theorem 3.1

Let $T\in\mathbb{N}$ and for every $t\in[T]$ let $X_t\in\mathbb{R}^{n_t\times d}$ and $y_t\in\mathbb{R}^{n_t}$. Define $\bar{y}=\sup_{t\in[T]}\left\lVert y_t\right\rVert_2$ and consider the multivaw_eq algorithm ran with regularization matrices satisfying: For all $\theta\in\mathbb{R}^d$, we have: where $A_T$ is defined by multivaw_close_eq_2.

Figures (3)

  • Figure 1: An example of hierarchical time series.
  • Figure 2: M5 dataset
  • Figure 3: Labour dataset

Theorems & Definitions (21)

  • Example 2.1: Univariate online linear regression
  • Remark 2.2: Equivalent problems
  • Remark 2.3: Links with OCO
  • Example 2.4: Online multiple-output regression
  • Example 2.5: Online state estimation
  • Theorem 3.1
  • Corollary 3.2
  • Example 4.1
  • Proposition 4.2
  • Proposition 4.3
  • ...and 11 more