Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robustifying Empirical Bayes

Roger Koenker, Jiaying Gu

Abstract

Two strategies are explored for robustifying classical denoising procedures for the Gaussian sequence model. First, the Hodges and Lehmann (1952) restricted Bayes approach is used to reduce sensitivity to the specification of the initial prior distribution. Second, alternatives to the Gaussian noise assumption are explored. In both cases proposals of Huber (1964) and Mallows (1978) play a crucial role.

Robustifying Empirical Bayes

Abstract

Two strategies are explored for robustifying classical denoising procedures for the Gaussian sequence model. First, the Hodges and Lehmann (1952) restricted Bayes approach is used to reduce sensitivity to the specification of the initial prior distribution. Second, alternatives to the Gaussian noise assumption are explored. In both cases proposals of Huber (1964) and Mallows (1978) play a crucial role.
Paper Structure (9 sections, 13 theorems, 107 equations, 4 figures, 4 tables)

This paper contains 9 sections, 13 theorems, 107 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Bayes Risk and Brown's Identity
  3. Restricted Bayes Solutions
  4. Restricted Empirical Bayes rules
  5. Some simulation experience
  6. Robustified Gaussian Likelihoods
  7. Some Simulation Experience
  8. conclusion
  9. Computation of the Mallows least favorable distribution

Key Result

Theorem 3.1

Provided the NPMLE estimator of the marginal density $f_{\hat{G}}$ is Hellinger consistent for the true marginal density $f_{G_0}$, then as $n \to \infty$,

Figures (4)

  • Figure 5: The left panel of the figure contrasts the NPMLE prior with the modified Mallows prior. The right panel contrasts the Huber and Mallows forms of the restricted Hodges-Lehmann decision rules based on the initial Kiefer-Wolfowitz NPMLE prior: the Huber rule is almost linear in the center while the Mallows rule is smoother near zero and oscillates around the Huber rule in the tails. In this example we take $\epsilon = 0.1$ on the presumption that the initial empirical prior is more reliable than in the prior examples.
  • Figure 6: The Huber and Mallows decision rules are contrasted with the Gaussian rule in a setting with $G = U[0,3]$. The heavier tail behavior of the Huber and Mallows base distribution, $\varphi$ results in more aggressive shrinkage with extreme observations discounted as the consequence of noise rather than signal. The Huber and Mallows rules both set $\epsilon = 0.1$ for this figure.
  • Figure 7: Mallows least favorable marginal density, probability mass function of the Mallows mixing distribution and log mass of the mixing distribution as a function of location indicating the approximate exponentiality of the mixing distribution as conjectured by Mallows.
  • Figure 8: Mass points of the Mallows contamination distribution $H^*$ with $G_0 = 0.5 \delta_{-2} + 0.5 \delta_2$. The solid black curve depicts the functional derivative of the objective function of the constrained Mallows's problem. The vertical green lines depict the location of the mass points of the solution, while their length represents the amount of mass assigned to each. In accordance with the Tukey "hanging rootogram" principle these lengths are rescaled as the square root of the respective masses. The dashed horizontal line represents the bound on the pointwise risk imposed by the Hodges-Lehmann constraint in this case approximately 1.67 induced by the choice of $\epsilon = 0.2$.

Theorems & Definitions (13)

  • Theorem 3.1
  • Theorem C.1
  • Lemma C.1
  • Lemma C.2
  • Lemma C.3: $\Gamma$-convergence in weak $L^2$
  • Theorem C.2
  • Lemma C.4: Tightness from weak $L^2$ convergence to a density
  • Lemma C.5
  • Lemma C.6
  • Lemma C.7
  • ...and 3 more