Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Mean Estimation under Quantization

Pedro Abdalla, Junren Chen

TL;DR

The paper addresses robust mean estimation from quantized data under distributed and adversarial conditions. It introduces one-bit and partial-quantization schemes, leveraging random dithering and a novel connection to robust statistics to achieve near-minimax rates, including in high dimensions. The authors also integrate robust mean estimators (e.g., Depersin–Lecue) to handle adversarial corruption and develop a partial-quantization regime that removes dependence on the mean's location while preserving near-optimal bit usage. The results offer practical, memory-efficient approaches for distributed learning with robustness guarantees, supported by numerical experiments. Together, the work advances theory and practice for robust, quantized mean estimation in challenging data environments.

Abstract

We consider the problem of mean estimation under quantization and adversarial corruption. We construct multivariate robust estimators that are optimal up to logarithmic factors in two different settings. The first is a one-bit setting, where each bit depends only on a single sample, and the second is a partial quantization setting, in which the estimator may use a small fraction of unquantized data.

Robust Mean Estimation under Quantization

TL;DR

The paper addresses robust mean estimation from quantized data under distributed and adversarial conditions. It introduces one-bit and partial-quantization schemes, leveraging random dithering and a novel connection to robust statistics to achieve near-minimax rates, including in high dimensions. The authors also integrate robust mean estimators (e.g., Depersin–Lecue) to handle adversarial corruption and develop a partial-quantization regime that removes dependence on the mean's location while preserving near-optimal bit usage. The results offer practical, memory-efficient approaches for distributed learning with robustness guarantees, supported by numerical experiments. Together, the work advances theory and practice for robust, quantized mean estimation in challenging data environments.

Abstract

We consider the problem of mean estimation under quantization and adversarial corruption. We construct multivariate robust estimators that are optimal up to logarithmic factors in two different settings. The first is a one-bit setting, where each bit depends only on a single sample, and the second is a partial quantization setting, in which the estimator may use a small fraction of unquantized data.
Paper Structure (11 sections, 9 theorems, 117 equations, 5 figures)

This paper contains 11 sections, 9 theorems, 117 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Background on robust statistics and dithering
  4. Mean Estimation in the Distributed Setting
  5. The One-Dimensional Case
  6. The Multivariate Case
  7. Mean Estimation under Partial Quantization
  8. The One-Dimensional Case
  9. The Adversarial Noise in the One-Dimensional Case
  10. The Multivariate Case
  11. Numerical Examples

Key Result

Theorem 3.1

There exists an absolute constant $C>0$ for which the following holds. The estimator $\widehat{\mu}$ given by eq:quantization_1dim_1bit and eq:estimator_1dim_1bit satisfies with probability at least $1-\delta$, Moreover, under the pre-quantization noise eq:adversarial_noise_pre or the post-quantization noise eq:adversarial_noise_pos, the estimator satisfies, with probability $1-\delta$,

Figures (5)

  • Figure 1: Corruption-free univariate mean estimation.
  • Figure 2: Corruption-free multivariate mean estimation under isotropic covariance.
  • Figure 3: Multivariate mean estimation with low trace $\mathop{\mathrm{Tr}}\nolimits(\Sigma)$.
  • Figure 4: Robust univariate mean estimation.
  • Figure 5: Empirical mean does not enjoy strong robustness.

Theorems & Definitions (18)

  • Theorem 3.1
  • proof
  • Theorem 3.2: Multivariate without corruption
  • proof
  • Remark 3.3
  • Proposition 3.4: Depersin and Lecuédepersin2022robust
  • Theorem 3.5: Multivariate with corruption
  • proof
  • Theorem 4.1
  • proof
  • ...and 8 more