Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compound decisions and empirical Bayes via Bayesian nonparametrics

Nikolaos Ignatiadis, Sid Kankanala

TL;DR

It is shown that a Dirichlet-process-based Bayesian procedure achieves near-optimal regret bounds and also provides parallel guarantees under a hierarchical model in which the means are drawn from a true unknown prior distribution.

Abstract

We study the Gaussian sequence compound decision problem and analyze a Bayesian nonparametric estimator from an empirical Bayes, regret-based perspective. Motivated by sharp results for the classical nonparametric maximum likelihood estimator (NPMLE), we ask whether an analogous guarantee can be obtained using a standard Bayesian nonparametric prior. We show that a Dirichlet-process-based Bayesian procedure achieves near-optimal regret bounds. Our main results are stated in the compound decision framework, where the mean vector is treated as fixed, while we also provide parallel guarantees under a hierarchical model in which the means are drawn from a true unknown prior distribution. The posterior mean Bayes rule is, a fortiori, admissible, whereas we show that the NPMLE plug-in rule is inadmissible.

Compound decisions and empirical Bayes via Bayesian nonparametrics

TL;DR

It is shown that a Dirichlet-process-based Bayesian procedure achieves near-optimal regret bounds and also provides parallel guarantees under a hierarchical model in which the means are drawn from a true unknown prior distribution.

Abstract

We study the Gaussian sequence compound decision problem and analyze a Bayesian nonparametric estimator from an empirical Bayes, regret-based perspective. Motivated by sharp results for the classical nonparametric maximum likelihood estimator (NPMLE), we ask whether an analogous guarantee can be obtained using a standard Bayesian nonparametric prior. We show that a Dirichlet-process-based Bayesian procedure achieves near-optimal regret bounds. Our main results are stated in the compound decision framework, where the mean vector is treated as fixed, while we also provide parallel guarantees under a hierarchical model in which the means are drawn from a true unknown prior distribution. The posterior mean Bayes rule is, a fortiori, admissible, whereas we show that the NPMLE plug-in rule is inadmissible.
Paper Structure (34 sections, 20 theorems, 170 equations, 2 tables)

This paper contains 34 sections, 20 theorems, 170 equations, 2 tables.

Table of Contents

  1. Introduction
  2. Statistical setting and preview of main result
  3. Structure of this paper
  4. Related work
  5. Empirical Bayes via Bayesian nonparametrics
  6. Empirical Bayes regret
  7. Bayes properties and admissibility
  8. Posterior contraction and proof of Theorem \ref{['theo:regret_bayes']}
  9. Compound decisions via Bayesian nonparametrics
  10. Compound decisions regret
  11. Bayes properties and admissibility
  12. Posterior contraction and proof of Theorem \ref{['theo:main_risk_bound_eq']}
  13. I. J. Good's staircase
  14. Numerical results
  15. Remarks on implementation
  16. ...and 19 more sections

Key Result

Theorem 1.2

Suppose that Specification assu:DP holds for $\Pi$. Then there exists a constant $C>0$ depending only on $(M, \alpha, \eta)$ such that for all $n \geq 2$, where the infimum is over all possible priors $\widetilde{\Pi}$ and $R(\cdot,\boldsymbol{\mu})$ is defined as the frequentist risk in eq:RMSE when data is generated according to eq:hierarchy1 with $\boldsymbol{\mu}$.

Theorems & Definitions (39)

  • Theorem 1.2
  • Theorem 3.1
  • Proposition 3.2: LOO Representation
  • Definition 3.3: Admissibility
  • Proposition 3.4
  • Proposition 3.5
  • Theorem 3.6: Posterior contraction
  • Lemma 3.7
  • proof : Proof of Theorem \ref{['theo:main_risk_bound_bayes']}
  • Theorem 4.1
  • ...and 29 more