Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maximum-Variance-Reduction Stratification for Improved Subsampling

Dingyi Wang, Haiying Wang, Qingpei Hu

TL;DR

The paper tackles the computational challenge of performing M-estimation on massive datasets by augmenting existing non-uniform subsampling with Maximum-Variance-Reduction Stratification (MVRS). MVRS selects a stratification variable that maximizes variance reduction, implemented via a two-step pilot-and-stratify procedure with a linear-time update, and yields an estimator with asymptotic normality and reduced variance relative to unstratified schemes. The authors provide a practical algorithm, a feasible variance estimator, and theoretical guarantees, showing substantial variance reductions in both synthetic GLMs (logistic and Poisson) and a real SUSY dataset. The approach is shown to be near-optimal compared with fully ranked stratification while maintaining scalable computation, making it attractive for scalable inference in big-data settings.

Abstract

Subsampling is a widely used and effective approach for addressing the computational challenges posed by massive datasets. Substantial progress has been made in developing non-uniform, probability-based subsampling schemes that prioritize more informative observations. We propose a novel stratification mechanism that can be combined with existing subsampling designs to further improve estimation efficiency. We establish the estimator's asymptotic normality and quantify the resulting efficiency gains, which enables a principled procedure for selecting stratification variables and interval boundaries that target reductions in asymptotic variance. The resulting algorithm, Maximum-Variance-Reduction Stratification (MVRS), achieves significant improvements in estimation efficiency while incurring only linear additional computational cost. MVRS is applicable to both non-uniform and uniform subsampling methods. Experiments on simulated and real datasets confirm that MVRS markedly reduces estimator variance and improves accuracy compared with existing subsampling methods.

Maximum-Variance-Reduction Stratification for Improved Subsampling

TL;DR

The paper tackles the computational challenge of performing M-estimation on massive datasets by augmenting existing non-uniform subsampling with Maximum-Variance-Reduction Stratification (MVRS). MVRS selects a stratification variable that maximizes variance reduction, implemented via a two-step pilot-and-stratify procedure with a linear-time update, and yields an estimator with asymptotic normality and reduced variance relative to unstratified schemes. The authors provide a practical algorithm, a feasible variance estimator, and theoretical guarantees, showing substantial variance reductions in both synthetic GLMs (logistic and Poisson) and a real SUSY dataset. The approach is shown to be near-optimal compared with fully ranked stratification while maintaining scalable computation, making it attractive for scalable inference in big-data settings.

Abstract

Subsampling is a widely used and effective approach for addressing the computational challenges posed by massive datasets. Substantial progress has been made in developing non-uniform, probability-based subsampling schemes that prioritize more informative observations. We propose a novel stratification mechanism that can be combined with existing subsampling designs to further improve estimation efficiency. We establish the estimator's asymptotic normality and quantify the resulting efficiency gains, which enables a principled procedure for selecting stratification variables and interval boundaries that target reductions in asymptotic variance. The resulting algorithm, Maximum-Variance-Reduction Stratification (MVRS), achieves significant improvements in estimation efficiency while incurring only linear additional computational cost. MVRS is applicable to both non-uniform and uniform subsampling methods. Experiments on simulated and real datasets confirm that MVRS markedly reduces estimator variance and improves accuracy compared with existing subsampling methods.
Paper Structure (18 sections, 8 theorems, 60 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 18 sections, 8 theorems, 60 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Subsampling Framework and Benefit of Stratification
  3. Non-uniform Subsampling Framework
  4. Asymptotics for Stratified Subsampling Estimators
  5. Maximum-Variance-Reduction Stratification
  6. Stratification Variable
  7. Stratification Interval Boundaries
  8. Practical Algorithm
  9. Theoretical property and variance estimation for the practical estimator
  10. Simulation
  11. Parameter Estimation
  12. Effect of the strata number $k$
  13. MSE Estimation
  14. Real Data Example
  15. Conclusion and Discussion
  16. ...and 3 more sections

Key Result

Proposition 1

Under the Assumptions asmp01-asmp05 in Section sec:weight-strat-subs, as $N \rightarrow \infty$ and $n \rightarrow \infty$, the estimator ${\hat{{\theta}}}_{n}^{{\mathrm{sub}}}$ satisfies in distribution, where The notation $^{\otimes2}$ denotes the outer product, i.e., $\dot{l}^{\otimes2}=\dot{l}\dot{l}^{\rm T}$, and $\dot{l}$ and $\ddot{l}$ are the gradient vector and Hessian matrix of $l$ wit

Figures (6)

  • Figure 1: Results of logistic regression
  • Figure 2: Results of poisson regression
  • Figure 3: Effect of Strata Number, $n=200$
  • Figure 4: Effect of Strata Number, $n=1000$
  • Figure 5: Results of Empirical and Estimated MSEs
  • ...and 1 more figures

Theorems & Definitions (15)

  • Proposition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • ...and 5 more