Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Mean Estimation with Coded Relayed Observations

Yan Hao Ling, Zhouhao Yang, Jonathan Scarlett

TL;DR

The paper studies minimax mean estimation with coded relayed observations, deriving tight large-deviations error exponents for mean estimation when a teacher observes samples and conveys information through a memoryless channel to a student. It introduces a block-structured teaching strategy and a hypothesis-testing based decoder to achieve exponents that match or nearly match the corresponding direct-observation benchmarks, across Bernoulli/BSC and generalized sub-Gaussian/discrete memoryless channel settings. The main results show that, for Bernoulli sources, the optimal exponent scales as $E^* \ge \min\big(E^{src}_{\varepsilon}, E^{chan}(0)\big)$ (up to multiplicative constants in the converse), while for sub-Gaussian sources one has $E^* \ge \min\big(\tfrac{\varepsilon^2}{2\sigma^2}, E^{chan}(0)\big)$, with Gaussian and heavy-tailed extensions discussed. These findings provide fundamental limits for estimation under communication constraints and offer polynomial-time protocols with strong asymptotic guarantees for a range of source and channel models, including extensions to vector-valued and heavier-tailed settings.

Abstract

We consider a problem of statistical mean estimation in which the samples are not observed directly, but are instead observed by a relay (``teacher'') that transmits information through a memoryless channel to the decoder (``student''), who then produces the final estimate. We consider the minimax estimation error in the large deviations regime, and establish achievable error exponents that are tight in broad regimes of the estimation accuracy and channel quality. In contrast, two natural baseline methods are shown to yield strictly suboptimal error exponents. We initially focus on Bernoulli sources and binary symmetric channels, and then generalize to sub-Gaussian and heavy-tailed settings along with arbitrary discrete memoryless channels.

Statistical Mean Estimation with Coded Relayed Observations

TL;DR

The paper studies minimax mean estimation with coded relayed observations, deriving tight large-deviations error exponents for mean estimation when a teacher observes samples and conveys information through a memoryless channel to a student. It introduces a block-structured teaching strategy and a hypothesis-testing based decoder to achieve exponents that match or nearly match the corresponding direct-observation benchmarks, across Bernoulli/BSC and generalized sub-Gaussian/discrete memoryless channel settings. The main results show that, for Bernoulli sources, the optimal exponent scales as (up to multiplicative constants in the converse), while for sub-Gaussian sources one has , with Gaussian and heavy-tailed extensions discussed. These findings provide fundamental limits for estimation under communication constraints and offer polynomial-time protocols with strong asymptotic guarantees for a range of source and channel models, including extensions to vector-valued and heavier-tailed settings.

Abstract

We consider a problem of statistical mean estimation in which the samples are not observed directly, but are instead observed by a relay (``teacher'') that transmits information through a memoryless channel to the decoder (``student''), who then produces the final estimate. We consider the minimax estimation error in the large deviations regime, and establish achievable error exponents that are tight in broad regimes of the estimation accuracy and channel quality. In contrast, two natural baseline methods are shown to yield strictly suboptimal error exponents. We initially focus on Bernoulli sources and binary symmetric channels, and then generalize to sub-Gaussian and heavy-tailed settings along with arbitrary discrete memoryless channels.
Paper Structure (41 sections, 17 theorems, 54 equations, 4 figures)

This paper contains 41 sections, 17 theorems, 54 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Discussion on large deviations and other regimes
  4. Discussion of protocols
  5. Non-causal setting
  6. One-shot estimate-and-forward
  7. Simple forwarding
  8. Existing 1-bit teaching and learning protocol
  9. Our approach
  10. Summary of our results
  11. Preliminaries
  12. Distance measures and distance-based codes
  13. Channel coding error exponents
  14. Bernoulli Source and Binary Symmetric Channel
  15. Numerical comparisons
  16. ...and 26 more sections

Key Result

Lemma 3

In the non-causal setting, for any distribution class $\mathcal{P}_X$ whose set of possible means is $\Theta^* = [0,1]$,This assumption is trivial for the Bernoulli class, and we will discuss in Section sec:discussion how it can be relaxed in other cases. and any $\varepsilon \in (0,\frac{1}{2})$, w

Figures (4)

  • Figure 1: Illustration of our problem setup; the quantity being estimated is $\theta^* = \mathbb{E}[X]$.
  • Figure 2: Comparison of error exponents in the Bernoulli+BSC setting with crossover probability $p = 0.1$. (The jagged behavior of the curve 'Non-Causal' for higher $\varepsilon$ comes from requiring an integer number of codewords and that integer becoming small.)
  • Figure 3: Comparison of error exponents in the Bernoulli+BSC setting with accuracy parameter $\varepsilon = 0.1$.
  • Figure 4: A diagrammatic representation of the block protocol

Theorems & Definitions (31)

  • Definition 1
  • Definition 2
  • Lemma 3
  • proof
  • Lemma 4
  • proof
  • Lemma 5
  • Lemma 6
  • proof
  • Lemma 7
  • ...and 21 more