Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparsified-Learning for Heavy-Tailed Locally Stationary Processes

Yingjie Wang, Mokhtar Z. Alaya, Salim Bouzebda, Xinsheng Liu

TL;DR

This work addresses sparse learning for high-dimensional, heavy-tailed locally stationary processes by integrating a kernel-based additive modeling framework with sparsity-inducing penalties. It develops non-asymptotic concentration results for locally stationary $\beta$-mixing sequences with sub-Weibull and regularly varying tails and derives oracle inequalities under slow and fast rates for both $\ell_1$ (Lasso) and weighted total variation penalties. The results hinge on tail indices, mixing rates, bandwidth choices, and restricted eigenvalue-type conditions, yielding robust prediction guarantees in nonstationary, heavy-tailed settings. Overall, the paper extends sparse learning theory to nonstationary, heavy-tailed time series, providing practical estimation procedures with finite-sample performance guarantees.

Abstract

Sparsified Learning is ubiquitous in many machine learning tasks. It aims to regularize the objective function by adding a penalization term that considers the constraints made on the learned parameters. This paper considers the problem of learning heavy-tailed LSP. We develop a flexible and robust sparse learning framework capable of handling heavy-tailed data with locally stationary behavior and propose concentration inequalities. We further provide non-asymptotic oracle inequalities for different types of sparsity, including $\ell_1$-norm and total variation penalization for the least square loss.

Sparsified-Learning for Heavy-Tailed Locally Stationary Processes

TL;DR

This work addresses sparse learning for high-dimensional, heavy-tailed locally stationary processes by integrating a kernel-based additive modeling framework with sparsity-inducing penalties. It develops non-asymptotic concentration results for locally stationary -mixing sequences with sub-Weibull and regularly varying tails and derives oracle inequalities under slow and fast rates for both (Lasso) and weighted total variation penalties. The results hinge on tail indices, mixing rates, bandwidth choices, and restricted eigenvalue-type conditions, yielding robust prediction guarantees in nonstationary, heavy-tailed settings. Overall, the paper extends sparse learning theory to nonstationary, heavy-tailed time series, providing practical estimation procedures with finite-sample performance guarantees.

Abstract

Sparsified Learning is ubiquitous in many machine learning tasks. It aims to regularize the objective function by adding a penalization term that considers the constraints made on the learned parameters. This paper considers the problem of learning heavy-tailed LSP. We develop a flexible and robust sparse learning framework capable of handling heavy-tailed data with locally stationary behavior and propose concentration inequalities. We further provide non-asymptotic oracle inequalities for different types of sparsity, including -norm and total variation penalization for the least square loss.

Paper Structure

This paper contains 36 sections, 23 theorems, 285 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related works.
  3. Contributions.
  4. Layout of the paper.
  5. Notation.
  6. Background on heavy-tailed LSP
  7. Locally stationary processes
  8. Heavy-tailed distribution
  9. Sparse penalized estimation procedure
  10. Block sparsity
  11. Assumptions
  12. Concentration inequalities for heavy-tailed LSP
  13. Oracle inequalities
  14. Slow rates
  15. Sub-Weibull distribution
  16. ...and 21 more sections

Key Result

Proposition 1

Let $\{\varepsilon_{t,T}\}_{t=1}^{T}$ follows the sub-Weibull distribution with constant $(\eta_{2},C_{\varepsilon})$, the sequence $\{W_{t,r,T}^{j}\}_{t}$ defined in (equation-W_t,T). Assumption Ass:1-Ass:KB2 are satisfied. Let $h=\mathcal{O}(T^{-\xi})$ with $0<\xi<\frac{1}{2}$, for any $\gamma> 2C where $1/\eta = 1/\eta_{1} + 1/\eta_{2}$, $\eta < 1$, $C_{K}=C_{K_{1}}C_{K_{2}}$, the constant $C_{

Figures (2)

  • Figure 1: Sub-Weibull distribution with $C=1$.
  • Figure 2: Pareto distribution with $u=1$.

Theorems & Definitions (37)

  • Definition 1: Locally stationary process, see VOGT2012
  • Remark 1
  • Example 1
  • Definition 2: Mixing condition, see bradley2005
  • Definition 3: Tail-capturing distribution
  • Example 2: Sub-Weibull distribution
  • Definition 4: Regularly varying tail distribution
  • Example 3: Pareto distribution
  • Definition 5
  • Remark 2
  • ...and 27 more