Function estimation in the empirical Bayes setting
Benjamin Kang, Yury Polyanskiy, Anzo Teh
TL;DR
The paper studies function estimation in empirical Bayes for Poisson and normal means, focusing on estimating posterior expectations of smooth functionals $ell( heta)$ given data. It develops a unifying view that smooth functionals admit faster minimax rates than full-prior recovery, with tight bounds for polynomial functionals in the Poisson model and general smoothness results; Tweedie-style posterior formulas enable practical estimators such as Robbins-, NPMLE-, and ERM-based methods. The authors establish matching upper and lower bounds, reveal a precise approximation-theoretic origin for rate gains (low-degree polynomial approximability of smooth functionals), and connect nonparametric EB theory with polynomial approximation and statistical inverse problems. They also show a transport-map perspective explaining why large $W_1$ distances between $ pi$ and $ ext{pi}$ do not preclude accurate posterior functionals, and they extend techniques like offset Rademacher complexity to this EB functional-estimation setting. The results delineate a sharp hierarchy—from slow deconvolution in the worst case to near-parametric convergence for smooth posterior functionals—providing a deeper understanding of when and why empirical Bayes methods succeed in functional estimation. This work thereby advances nonparametric EB theory and offers concrete, minimax-optimal estimators for key EB functionals under Poisson and normal-model channels.
Abstract
We study function estimation in the empirical Bayes setting for Poisson and normal means. Specifically, given observations $Y_i\sim f(\cdot; θ_i)$ with latent parameters $θ_i\sim π$, the goal is to estimate $\mathbb{E}_π[\ell(θ)|X = x]$. This task lies between classical deconvolution (recovering the full prior $π$), and standard empirical Bayes mean estimation. While the minimax risk for estimating $π$ in the Wasserstein distance is known to decay only logarithmically, we show that estimating certain smooth functions admits dramatically faster rates. In particular, for polynomial functions of degree $k$ in the Poisson model, we establish a tight bound of $Θ(\frac{1}{n}(\frac{\log n}{\log \log n})^{k+1})$ and $Θ(\frac{1}{n}(\log n)^{2k+1})$ for bounded and subexponential priors, respectively, attainable by estimators mimicking those that achieve optimal regret for the mean estimation problem (Robbins, mininum distance, ERM). Our analysis identifies the approximation-theoretic origin of this improvement: smooth functions can be well-approximated by low-degree polynomials, whereas Lipschitz functions require dense polynomial approximations, incurring a $\frac{1}{k}$ loss for degree $k$ polynomial approximation. The results reveal a sharp hierarchy in the difficulty of empirical Bayes problems: ranging from slow, logarithmic deconvolution to near-parametric convergence for smooth posterior functionals, and establish new connections between nonparametric empirical Bayes theory, polynomial approximation, and statistical inverse problems. Finally, we complement our analysis with a lower bound of $Ω(\frac 1n (\frac{\log n}{\log \log n})^{k+1})$ (bounded priors) and $Ω(\frac 1n (\log n)^{k + 1})$ (subgaussian priors) for the normal means model.