Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$L_q$ Lower Bounds on Distributed Estimation via Fisher Information

Wei-Ning Chen, Ayfer Özgür

TL;DR

This work generalizes the van Trees inequality to general $L_q$ loss by leveraging Efroimovich's entropy bound, enabling non- L_2 minimax analysis. It then applies the generalized bound to distributed estimation under communication and local differential privacy constraints, deriving global minimax lower bounds for Gaussian mean/covariance, discrete distribution, and product Bernoulli models, under $b$-bit and DP constraints, with explicit rates. The paper also develops local, non-asymptotic minimax bounds for instance-specific hardness, notably for discrete distribution estimation, and provides nearly tight results (up to log factors) via achievable sample-splitting schemes. Overall, the results establish that Fisher-information based methods can yield tight $L_q$ lower bounds in information-constrained distributed settings and can produce local, non-asymptotic insights beyond the traditional squared loss framework.

Abstract

Van Trees inequality, also known as the Bayesian Cramér-Rao lower bound, is a powerful tool for establishing lower bounds for minimax estimation through Fisher information. It easily adapts to different statistical models and often yields tight bounds. Recently, its application has been extended to distributed estimation with privacy and communication constraints where it yields order-wise optimal minimax lower bounds for various parametric tasks under squared $L_2$ loss. However, a widely perceived drawback of the van Trees inequality is that it is limited to squared $L_2$ loss. The goal of this paper is to dispel that perception by introducing a strengthened version of the van Trees inequality that applies to general $L_q$ loss functions by building on the Efroimovich's inequality -- a lesser-known entropic inequality dating back to the 1970s. We then apply the generalized van Trees inequality to lower bound $L_q$ loss in distributed minimax estimation under communication and local differential privacy constraints. This leads to lower bounds for $L_q$ loss that apply to sequentially interactive and blackboard communication protocols. Additionally, we show how the generalized van Trees inequality can be used to obtain \emph{local} and \emph{non-asymptotic} minimax results that capture the hardness of estimating each instance at finite sample sizes.

$L_q$ Lower Bounds on Distributed Estimation via Fisher Information

TL;DR

This work generalizes the van Trees inequality to general loss by leveraging Efroimovich's entropy bound, enabling non- L_2 minimax analysis. It then applies the generalized bound to distributed estimation under communication and local differential privacy constraints, deriving global minimax lower bounds for Gaussian mean/covariance, discrete distribution, and product Bernoulli models, under -bit and DP constraints, with explicit rates. The paper also develops local, non-asymptotic minimax bounds for instance-specific hardness, notably for discrete distribution estimation, and provides nearly tight results (up to log factors) via achievable sample-splitting schemes. Overall, the results establish that Fisher-information based methods can yield tight lower bounds in information-constrained distributed settings and can produce local, non-asymptotic insights beyond the traditional squared loss framework.

Abstract

Van Trees inequality, also known as the Bayesian Cramér-Rao lower bound, is a powerful tool for establishing lower bounds for minimax estimation through Fisher information. It easily adapts to different statistical models and often yields tight bounds. Recently, its application has been extended to distributed estimation with privacy and communication constraints where it yields order-wise optimal minimax lower bounds for various parametric tasks under squared loss. However, a widely perceived drawback of the van Trees inequality is that it is limited to squared loss. The goal of this paper is to dispel that perception by introducing a strengthened version of the van Trees inequality that applies to general loss functions by building on the Efroimovich's inequality -- a lesser-known entropic inequality dating back to the 1970s. We then apply the generalized van Trees inequality to lower bound loss in distributed minimax estimation under communication and local differential privacy constraints. This leads to lower bounds for loss that apply to sequentially interactive and blackboard communication protocols. Additionally, we show how the generalized van Trees inequality can be used to obtain \emph{local} and \emph{non-asymptotic} minimax results that capture the hardness of estimating each instance at finite sample sizes.
Paper Structure (23 sections, 14 theorems, 39 equations)

This paper contains 23 sections, 14 theorems, 39 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Information-Constrained Estimation
  4. Problem Setup
  5. Global Minimax Lower Bounds
  6. Achievability
  7. Local Minimax Bounds
  8. $b$-bit communication constraint
  9. $\varepsilon$-local differential privacy
  10. Conclusion
  11. Global Minimax Lower Bounds under LDP
  12. Missing Proofs
  13. Proof of Theorem \ref{['thm:generalized_van_trees']}
  14. Proof of Theorem \ref{['thm:local_b_bit_lower_bound']}
  15. Constructing the sub-model $\Theta^h_p$.
  16. ...and 8 more sections

Key Result

Theorem 2.2

Let $X \sim P_\theta$ and $\theta \sim \pi$ for some prior distribution $\pi$. Let Assumption assumption:regularity hold. Then, for any prior distribution $\pi$ on $\Theta$ for which the information theorist's Fisher information $J(\pi)$ exists, it holds that

Theorems & Definitions (16)

  • Theorem 2.2: van2004detectiongill1995applications
  • Theorem 2.3: efroimovich1980information
  • Theorem 2.4: Generalized van Trees Inequality
  • Proposition 2.5
  • Remark 3.1
  • Corollary 3.2: Estimation under $b$-bit constraint
  • Lemma 3.3
  • Theorem 4.1: $L_2$ local minimax error chen2021pointwise
  • Theorem 4.2: $b$-bit $L_q$ local minimax lower bounds
  • Remark 4.3
  • ...and 6 more