Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Smoothed SGD for quantiles: Bahadur representation and Gaussian approximation

Likai Chen, Georg Keilbar, Wei Biao Wu

TL;DR

The paper addresses online quantile estimation in streaming data where exact sample quantiles are memory-intensive. It introduces a smoothed SGD algorithm that replaces the non-smooth quantile indicator with a Lipschitz smoothed score, ensuring monotone quantile curves and enabling online updates. The authors derive non-asymptotic tail bounds for both averaged and non-averaged variants, establish a uniform Bahadur representation, and prove a Gaussian approximation that supports simultaneous inference across dimensions and quantile levels. Simulation studies validate the theory, showing good finite-sample performance and reliable uniform confidence-band construction.

Abstract

This paper considers the estimation of quantiles via a smoothed version of the stochastic gradient descent (SGD) algorithm. By smoothing the score function in the conventional SGD quantile algorithm, we achieve monotonicity in the quantile level in that the estimated quantile curves do not cross. We derive non-asymptotic tail probability bounds for the smoothed SGD quantile estimate both for the case with and without Polyak-Ruppert averaging. For the latter, we also provide a uniform Bahadur representation and a resulting Gaussian approximation result. Numerical studies show good finite sample behavior for our theoretical results.

Smoothed SGD for quantiles: Bahadur representation and Gaussian approximation

TL;DR

The paper addresses online quantile estimation in streaming data where exact sample quantiles are memory-intensive. It introduces a smoothed SGD algorithm that replaces the non-smooth quantile indicator with a Lipschitz smoothed score, ensuring monotone quantile curves and enabling online updates. The authors derive non-asymptotic tail bounds for both averaged and non-averaged variants, establish a uniform Bahadur representation, and prove a Gaussian approximation that supports simultaneous inference across dimensions and quantile levels. Simulation studies validate the theory, showing good finite-sample performance and reliable uniform confidence-band construction.

Abstract

This paper considers the estimation of quantiles via a smoothed version of the stochastic gradient descent (SGD) algorithm. By smoothing the score function in the conventional SGD quantile algorithm, we achieve monotonicity in the quantile level in that the estimated quantile curves do not cross. We derive non-asymptotic tail probability bounds for the smoothed SGD quantile estimate both for the case with and without Polyak-Ruppert averaging. For the latter, we also provide a uniform Bahadur representation and a resulting Gaussian approximation result. Numerical studies show good finite sample behavior for our theoretical results.
Paper Structure (15 sections, 7 theorems, 127 equations, 5 figures, 3 tables)

This paper contains 15 sections, 7 theorems, 127 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The smoothed SGD algorithm
  3. Non-asymptotic tail probability bounds for smoothed SGD
  4. Bahadur representation and Gaussian approximation
  5. A Monte Carlo Study
  6. Conclusion
  7. Proofs for Results in Section \ref{['sec:tailprobability']}
  8. Proof of Theorem \ref{['thm:singletail']}
  9. Proof of Theorem \ref{['thm:tailsingleonerefine']}
  10. Proof of Theorem \ref{['thm:tail']}
  11. Proofs of Results in Section \ref{['sec:bahadur']}
  12. Proof of Theorem \ref{['thm:unifbahadur']}
  13. Proof of Theorem \ref{['thm:main']}
  14. Proof of Lemma \ref{['lem:diffxitilxi']}
  15. Additional Simulation Results

Key Result

Lemma 2.1

(Monotonicity) For any $1\leq i\leq p,$$\tau\geq \tau'$ and $k\in\mathbb{N},$ we have $Y_{i,k}(\tau)\geq Y_{i,k}(\tau').$

Figures (5)

  • Figure 1: Visualization of $g(x)$ (solid line) vs. the nonsmooth indicator function (dashed line).
  • Figure 2: Comparison of the SGD algorithm (left panel) and the smoothed SGD algorithm (right panel) for $\tau=0.4$ (black line), $\tau=0.5$ (red line) and $\tau=0.6$ (blue line).
  • Figure 3: QQ plots for known sparsity for Gaussian data (left panels) and $t_{10}$ data (right panel).
  • Figure 4: QQ plots for the performance of the Gaussian approximation result for conditional quantile estimation (estimated sparsity case).
  • Figure 5: QQ plots for estimated sparsity for Gaussian data (left panel) and $t_{10}$ data (right panel).

Theorems & Definitions (14)

  • Lemma 2.1
  • proof
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 4.1
  • Theorem 4.2
  • proof
  • proof
  • proof
  • ...and 4 more