SGD with Dependent Data: Optimal Estimation, Regret, and Inference

TL;DR

This work develops a unified analysis of stochastic gradient descent (SGD) when data are temporally dependent both in the covariates and noise, and when dependence is amplified by adaptive decision making in contextual Bandits. The authors establish non-asymptotic minimax-optimal rates for the final SGD iterate in estimation and regret, with tail bounds that hold for infinite horizons, and prove asymptotic Gaussianity with a $O_{\mathbb{P}}(1/\sqrt{t})$ remainder. They extend the framework to sparse online regression, introducing a conic decision-region approximation and an online algorithm with storage $O(d)$ that achieves long-run optimality and support recovery. For contextual bandits, they develop an SGD-based algorithm with flexible exploration schedules, derive a Bahadur-type representation for inference, and show that both estimation error and regret are minimax-optimal under mild dependence conditions. The results are supported by numerical experiments on linear regression, sparse regression, and linear bandits, demonstrating robustness to various dependence structures and confirming the practical viability of online estimation and inference under dependence.

Abstract

This work investigates the performance of the final iterate produced by stochastic gradient descent (SGD) under temporally dependent data. We consider two complementary sources of dependence: $(i)$ martingale-type dependence in both the covariate and noise processes, which accommodates non-stationary and non-mixing time series data, and $(ii)$ dependence induced by sequential decision making. Our formulation runs in parallel with classical notions of (local) stationarity and strong mixing, while neither framework fully subsumes the other. Remarkably, SGD is shown to automatically accommodate both independent and dependent information under a broad class of stepsize schedules and exploration rate schemes. Non-asymptotically, we show that SGD simultaneously achieves statistically optimal estimation error and regret, extending and improving existing results. In particular, our tail bounds remain sharp even for potentially infinite horizon $T=+\infty$. Asymptotically, the SGD iterates converge to a Gaussian distribution with only an $O_{\PP}(1/\sqrt{t})$ remainder, demonstrating that the supposed estimation-regret trade-off claimed in prior work can in fact be avoided. We further propose a new ``conic'' approximation of the decision region that allows the covariates to have unbounded support. For online sparse regression, we develop a new SGD-based algorithm that uses only $d$ units of storage and requires $O(d)$ flops per iteration, achieving the long term statistical optimality. Intuitively, each incoming observation contributes to estimation accuracy, while aggregated summary statistics guide support recovery.

Figures (5)

• Figure 1: Two approximations of the oracle decision region ${\mathcal{U}}_i^*$. The left panel illustrates the bounded polyhedral approximation $\widetilde{\mathcal{U}}_i$, shown as the shaded region enclosed by several line segments. The right panel shows the conic approximation $\mathcal{U}_i$, which forms an unbounded, scale-invariant region.
• Figure 2: Sensitivity with respect to $C_{a}$: SGD in online linear regression under both i.i.d. and dependent observations. Each curve depicts the convergence dynamics for $C_{a} \in \{ 3,10,50 \}$.
• Figure 3: Sensitivity with respect to $C_{b}$: SGD in online linear regression under both i.i.d. and dependent observations. Each curve shows the convergence dynamics for $C_{b} \in \{ 5,100,1000\}$.
• Figure 4: Sparse linear regression: the true parameter ${\boldsymbol{\beta}}^*\in{\mathbb R}^{99}$ has support size $|\operatorname{supp}({\boldsymbol{\beta}}^*)|=4$. The "Dense Alg" curve corresponds to the estimator in Theorem \ref{['thm:nonsymLR']}$(2)$; while the "$\alpha=7$" curve reports the estimation error from the algorithm in Theorem \ref{['thm:sparse-est-supprecv']}.
• Figure 5: Exploration rate schemes: estimation error and regret under three exploration schemes, $\pi_t=\tfrac{1}{2},\tfrac{5}{\sqrt{t-t_1+50}}$, and $\tfrac{5}{t-t_1+50}$, all computed under a common fixed SGD stepsize scheme.

Theorems & Definitions (64)

• Definition 1: Conditional Orlicz Norm
• Example 1
• Definition 2: Conditional Orlicz Norm of Random Vectors
• Example 2
• Example 3
• Theorem 1: Non-asymptotic Performance
• Remark 1: Tail Probability Accumulation
• Remark 2: Regret
• Theorem 2: Asymptotic Performance
• Lemma 1