Statistical Inference for Online Algorithms

TL;DR

This work addresses confidence inference for online estimators where full data passes and variance estimation are impractical. It introduces HulC, a variance-free, bucketed online inference approach that yields valid coordinate-wise confidence intervals from streaming outputs such as averaged SGD. Theoretical results show asymptotic validity under mild regularity, and numerical experiments in linear and logistic regression demonstrate that HulC achieves near nominal coverage with modest width inflation compared with Wald, plug-in, and t-stat methods, especially in challenging online and high-dimensional settings. The findings suggest HulC as a practical tool for real-time inference in streaming contexts, with broad applicability to online optimization and empirical risk minimization.

Abstract

Construction of confidence intervals and hypothesis tests for functionals based on asymptotically normal estimators is a classical topic in statistical inference. The simplest and in many cases optimal inference procedure is the Wald interval or the likelihood ratio test, both of which require an estimator and an estimate of the asymptotic variance of the estimator. Estimators obtained from online/sequential algorithms forces one to consider the computational aspects of the inference problem, i.e., one cannot access all of the data as many times as needed. Several works on this topic explored the online estimation of asymptotic variance. In this article, we propose computationally efficient, rate-optimal, and asymptotically valid confidence regions based on the output of online algorithms {\em without} estimating the asymptotic variance. As a special case, this implies inference from any algorithm that yields an asymptotically normal estimator. We focus our efforts on stochastic gradient descent with Polyak averaging to understand the practical performance of the proposed method.

Figures (21)

• Figure 1: For the linear regression task with identity covariance, $d=5$, $T=10^3$, and a small step size hyperparameter$c=0.01$, both the ASGD and SGD estimators for $\theta_{\infty}$ fail to converge. The distance from the target is worse for the larger coordinate, $e_5^{\top}\theta_{\infty}=1$, compared to the first coordinate, $e_1^{\top}\theta_{\infty}=0$, likely due to the initialization procedure, which favors smaller coordinates. In plot (c), which presents a histogram of $10,000$ repetitions of ASGD, there is no systematic bias: the mean ASGD estimates for $e_1^{\top}\theta_{\infty}=0$ is $-0.0006$. However, there is systematic bias for $e_5^{\top}\theta_{\infty}$ (as well as all larger coordinates), as seen in (d): the mean ASGD estimate for $e_5^{\top}\theta_{\infty}=1$ is $0.516$.
• Figure 2: For the linear regression task with identity covariance, $d=5$, $T=10^3$, and a large step size hyperparameter$c=2$, the SGD estimator tends to converge, but not the ASGD estimator, due to the large initial "wrong SGD points" at the start of the trajectory. The ASGD estimator is off by a large margin from the target parameter for both $e_1^{\top}\theta_{\infty}=0$ (a) and $e_5^{\top}\theta_{\infty}=1$ (b). In addition, like the case when $c$ is small (Figure \ref{['fig:ASGD_n1000_smallc_nonconvergence']}), there is systematic bias: in the histograms of $10^4$ replications, shown in plots (c) and (d), which both exclude the largest 1% outliers, the mean ASGD estimates for $e_1^{\top}\theta_{\infty}=0$ and $e_5^{\top}\theta_{\infty}=1$ are respectively $4.296$ and $4.687$.
• Figure 3: For the linear regression task with identity covariance, $d=5$, $T=10^3$, and a "mid-range" step size hyperparameter$c=0.5$, both the SGD and ASGD estimators tend to converge. The ASGD estimator appears unbiased: in the histograms of $10^4$ replications in plots (c) and (d), which both exclude the largest 1% outliers, the mean ASGD estimates for $e_1^{\top}\theta_{\infty}=0$ and $e_5^{\top}\theta_{\infty}=1$ are respectively $-0.0005$ and $0.998$.
• Figure 4: Comparison of Wald, AGSD plug-in, HulC, and t-stat methods in the linear regression setting with a Toeplitz covariance structure and dimension $d=5$. The ASGD plug-in confidence interval consistently demonstrates poor coverage, while both the HulC and the t-stat methods generally produce correct coverage for appropriately chosen $c$. Meanwhile, the width ratios for t-stat and HulC are not excessively large when $c$ is appropriately chosen; as the sample size $T$ increases, the ratios decrease.
• Figure 5: Comparison of Wald, AGSD plug-in, HulC, and t-stat methods in the logistic regression setting with a Toeplitz covariance structure and dimension $d=5$. Like the linear regression case (Figure \ref{['fig:linear_D5_Toeplitz_cov_wr']}), the ASGD plug-in confidence interval consistently demonstrates poor coverage, while the HulC and t-stat methods produce correct coverage for appropriately chosen $c$. The width ratios for t-stat and HulC are not excessively large when $c$ is appropriately chosen; as the sample size $T$ increases, the ratios decrease.

Theorems & Definitions (9)

• Theorem 1
• proof
• Corollary 1
• Theorem 2
• Theorem 3
• Corollary 2
• Remark : Asymptotic Normality of ASGD
• Lemma 1: Implication of \ref{['eq:ES-c']} and \ref{['eq:smooth-c']}
• proof