Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

Trambak Banerjee, Qiang Liu, Gourab Mukherjee, Wenguang Sun

Abstract

We develop a Nonparametric Empirical Bayes (NEB) framework for compound estimation in the discrete linear exponential family, which includes a wide class of discrete distributions frequently arising from modern big data applications. We propose to directly estimate the Bayes shrinkage factor in the generalized Robbins' formula via solving a scalable convex program, which is carefully developed based on a RKHS representation of the Stein's discrepancy measure. The new NEB estimation framework is flexible for incorporating various structural constraints into the data driven rule, and provides a unified approach to compound estimation with both regular and scaled squared error losses. We develop theory to show that the class of NEB estimators enjoys strong asymptotic properties. Comprehensive simulation studies as well as analyses of real data examples are carried out to demonstrate the superiority of the NEB estimator over competing methods.

A General Framework for Empirical Bayes Estimation in Discrete Linear Exponential Family

Abstract

We develop a Nonparametric Empirical Bayes (NEB) framework for compound estimation in the discrete linear exponential family, which includes a wide class of discrete distributions frequently arising from modern big data applications. We propose to directly estimate the Bayes shrinkage factor in the generalized Robbins' formula via solving a scalable convex program, which is carefully developed based on a RKHS representation of the Stein's discrepancy measure. The new NEB estimation framework is flexible for incorporating various structural constraints into the data driven rule, and provides a unified approach to compound estimation with both regular and scaled squared error losses. We develop theory to show that the class of NEB estimators enjoys strong asymptotic properties. Comprehensive simulation studies as well as analyses of real data examples are carried out to demonstrate the superiority of the NEB estimator over competing methods.

Paper Structure

This paper contains 28 sections, 11 theorems, 110 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. A General Framework for Compound Estimation in DLE Family
  3. Generalized Robbins' formula for DLE models
  4. Shrinkage estimation by convex optimization
  5. Structural constraints and bandwidth selection
  6. Theory
  7. Theoretical properties of the KSD measure
  8. Theoretical properties of $\hat{\pmb w}_n$
  9. Properties of the NEB estimator
  10. Numerical Results
  11. Implementation Details
  12. Simulations: Poisson Distribution
  13. Simulations: Binomial Distribution
  14. Real Data Analyses
  15. Estimation of Juvenile Delinquency rates
  16. ...and 13 more sections

Key Result

Lemma 1

Consider the DLE Model eq:DLEmodel. Let $p(\cdot)=\int p(\cdot|\theta)dG(\theta)$ be the marginal pmf of $Y$. Define for $k\in\{0,1\}$, Then the Bayes estimator that minimizes the risk $B_n^{(k)}(\bm \theta)$ is given by $\bm \delta^{\pi}_{(k)}=\{\delta_{(k),i}^{\pi}(y_i): 1\leq i\leq n\}$, where

Figures (5)

  • Figure 1: Poisson compound decision problem under scaled squared error loss: Risk estimates of the various estimators for scenarios 1 to 4.
  • Figure 2: Poisson compound decision problem under squared error loss: Risk estimates of the various estimators for scenarios 1 to 4.
  • Figure 3: Binomial compound decision problem under scaled squared error loss: Risk estimates of the various estimators for Scenarios 1 to 4.
  • Figure 4: Binomial compound decision problem under squared error loss: Risk estimates of the various estimators for Scenarios 1 to 4.
  • Figure 5: Observed Juvenile Delinquency rates in 2012. The top $500$ and bottom $500$ counties are plotted. The data on Florida arrests is not available in the us2012uniform database.

Theorems & Definitions (23)

  • Lemma 1: Generalized Robbins' formula
  • Remark 1
  • Definition 1A: NEB estimator
  • Definition 2A: ARE of ${\bm \delta}_{(1)}^{\sf neb}(\lambda)$ in the Poisson model
  • Definition 3A: ARE of ${\bm \delta}_{(1)}^{\sf neb}(\lambda)$ in the Binomial model
  • Remark 2
  • Remark 3
  • Remark 4
  • Theorem 1
  • Theorem 2A
  • ...and 13 more