Polynomial Log-Marginals and Tweedie's Formula : When Is Bayes Possible?
Jyotishka Datta, Nicholas G. Polson
TL;DR
The paper investigates when empirical Bayes rules that rely on the marginal density $m(y)$ can be derived from a formal Bayes model. It shows that if $\log m(y)$ is a polynomial of degree $K\ge 3$, no prior $F$ in an exponential family can yield $m(y)$ via Tweedie’s identity; only the quadratic case $K=2$ is compatible, with $F$ necessarily Gaussian. It further proves that a marginal $m$ is a Gaussian convolution only if, in a neighborhood of the smoothing parameter, it extends to a bounded solution of the heat equation, linking Bayes representation to the Hirschman–Widder/Weierstrass framework. These results delineate when practical EB procedures (e.g., NPMLE plug-ins or Lindsey’s method) align with formal Bayes rules, and emphasize that while observable-based updates achieve good risk profiles via SURE and regret considerations, they do not always correspond to valid Bayesian priors or posterior models.
Abstract
Motivated by Tweedie's formula for the Compound Decision problem, we examine the theoretical foundations of empirical Bayes estimators that directly model the marginal density $m(y)$. Our main result shows that polynomial log-marginals of degree $k \ge 3 $ cannot arise from any valid prior distribution in exponential family models, while quadratic forms correspond exactly to Gaussian priors. This provides theoretical justification for why certain empirical Bayes decision rules, while practically useful, do not correspond to any formal Bayes procedures. We also strengthen the diagnostic by showing that a marginal is a Gaussian convolution only if it extends to a bounded solution of the heat equation in a neighborhood of the smoothing parameter, beyond the convexity of $c(y)=\tfrac12 y^2+\log m(y)$.