Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Simple and Effective Random Forest Modelling for Nonlinear Time Series Data

Shihao Zhang, Zudi Lu, Chao Zheng

TL;DR

The paper tackles nonlinear time-series forecasting where temporal dependence complicates traditional random forests. It introduces RF-RW, a random-weight bootstrap-free forest that preserves serial dependence while injecting tree-level diversity through independent weights applied to the training series. The authors prove non-asymptotic concentration bounds and asymptotic uniform consistency for both fixed and high-dimensional feature spaces, supported by extensive simulations and an empirical UK COVID-19 daily cases application showing superior predictive accuracy. The work provides a practical, theoretically grounded approach for nonlinear time series and opens avenues for spatio-temporal extensions and quantile estimation.

Abstract

In this paper, we propose Random Forests by Random Weights (RF-RW), a theoretically grounded and practically effective alternative RF modelling for nonlinear time series data, where existing RF-based approaches struggle to adequately capture temporal dependence. RF-RW reconciles the strengths of classic RF with the temporal dependence inherent in time series forecasting. Specifically, it avoids the bootstrap resampling procedure, therefore preserves the serial dependence structure, whilst incorporates independent random weights to reduce correlations among trees. We establish non-asymptotic concentration bounds and asymptotic uniform consistency guarantees, for both fixed- and high-dimensional feature spaces, which extend beyond existing theoretical analyses of RF. Extensive simulation studies demonstrate that RF-RW outperforms existing RF-based approaches and other benchmarks such as SVM and LSTM. It also achieves the lowest error among competitors in our real-data example of predicting UK COVID-19 daily cases.

A Simple and Effective Random Forest Modelling for Nonlinear Time Series Data

TL;DR

The paper tackles nonlinear time-series forecasting where temporal dependence complicates traditional random forests. It introduces RF-RW, a random-weight bootstrap-free forest that preserves serial dependence while injecting tree-level diversity through independent weights applied to the training series. The authors prove non-asymptotic concentration bounds and asymptotic uniform consistency for both fixed and high-dimensional feature spaces, supported by extensive simulations and an empirical UK COVID-19 daily cases application showing superior predictive accuracy. The work provides a practical, theoretically grounded approach for nonlinear time series and opens avenues for spatio-temporal extensions and quantile estimation.

Abstract

In this paper, we propose Random Forests by Random Weights (RF-RW), a theoretically grounded and practically effective alternative RF modelling for nonlinear time series data, where existing RF-based approaches struggle to adequately capture temporal dependence. RF-RW reconciles the strengths of classic RF with the temporal dependence inherent in time series forecasting. Specifically, it avoids the bootstrap resampling procedure, therefore preserves the serial dependence structure, whilst incorporates independent random weights to reduce correlations among trees. We establish non-asymptotic concentration bounds and asymptotic uniform consistency guarantees, for both fixed- and high-dimensional feature spaces, which extend beyond existing theoretical analyses of RF. Extensive simulation studies demonstrate that RF-RW outperforms existing RF-based approaches and other benchmarks such as SVM and LSTM. It also achieves the lowest error among competitors in our real-data example of predicting UK COVID-19 daily cases.

Paper Structure

This paper contains 25 sections, 23 theorems, 144 equations, 8 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Random Forests by Random Weights
  3. Theoretical properties
  4. Transformation of explanatory features
  5. Assumptions
  6. Non-asymptotic concentration of RF-RW
  7. Consistency of fixed-dimensional RF-RW
  8. Consistency of high-dimensional RF-RW
  9. Simulation Studies
  10. An application to UK COVID-19 daily cases modelling
  11. Discussion
  12. Density of transformed feature and supporting lemmas
  13. Bounded density $h_Z$ of the transformed feature $Z$
  14. Supporting lemmas
  15. Proofs of results in Section \ref{['sec:theory_concentration']}
  16. ...and 10 more sections

Key Result

Theorem 3.1

Under Assumption assump:rfrw_all, there exists a positive constant $\beta$ such that the following two statements both hold with probability at least $1-6T^{-1}$ for all sufficiently large $T$:

Figures (8)

  • Figure 1: RMSTE of M1, including 3 lag features (underspecified), 5 lag features (correctly specified), 15 lag features (overspecified).
  • Figure 2: RMSTE of M2, including 2 lag features (underspecified), 3 lag features (correctly specified), 15 lag features (overspecified).
  • Figure 3: RMSTE of M3, including 6 lag features (underspecified), 8 lag features (correctly specified), 15 lag features (overspecified).
  • Figure 4: RF-RW-1 that includes 21 lags of the response. Left: Actual, Estimated, and Predicted cases against Date. Right: Actual and Predicted cases against Date.
  • Figure A1: RMSTE of M1 for RF-RW, with the random weights sampled from $\text{Exp}(1)$ (RF-RW-1, RF-RW-2), $\text{Lognormal}\left(\mu=-\log2/{2},\sigma^2=\log 2\right)$ (RF-RW-1-ln, RF-RW-2-ln), and $\text{Square root of Gamma}(\alpha=0.295,\theta=6.803)$ (RF-RW-1-sg, RF-RW-2-sg).
  • ...and 3 more figures

Theorems & Definitions (48)

  • Theorem 3.1
  • Remark 3.1
  • Definition 3.1: ($\xi,k,m$)-valid partitions
  • Theorem 3.2
  • Theorem 3.3
  • Definition 3.2: ($\tilde{\xi},k,m$)-valid partitions
  • Theorem 3.4
  • Lemma A.1
  • proof
  • Lemma A.2: Merlevède2009, Theorem 2
  • ...and 38 more