Excess Mean Squared Error of Empirical Bayes Estimators

TL;DR

This work develops a high-order framework to assess EB estimators for FIR models under Gaussian noise by introducing the excess MSE (XMSE), which captures the asymptotic MSE gap between EB and ML as $N\to\infty$. It derives an explicit XMSE expression for EB estimators with data-dependent hyper-parameters and specializes it to kernel-based regularized estimators using EB, SURE, and GCV hyper-parameter estimators. A finite-sample refinement is proposed to improve XMSE accuracy for moderate sample sizes, with numerical simulations demonstrating how kernel alignment with the true impulse response critically affects performance. The results offer a principled way to quantify and compare EB-based regularization techniques and guide kernel design for improved estimator accuracy in practice.

Abstract

Empirical Bayes estimators are based on minimizing the average risk with the hyper-parameters in the weighting function being estimated from observed data. The performance of an empirical Bayes estimator is typically evaluated by its mean squared error (MSE). However, the explicit expression for its MSE is generally unavailable for finite sample sizes. To address this issue, we define a high-order analytical criterion: the excess MSE. It quantifies the performance difference between the maximum likelihood and empirical Bayes estimators. An explicit expression for the excess MSE of an empirical Bayes estimator employing a general data-dependent hyper-parameter estimator is derived. As specific instances, we provide excess MSE expressions for kernel-based regularized estimators using the scaled empirical Bayes, Stein unbiased risk estimation, and generalized cross-validation hyper-parameter estimators. Moreover, we propose a modification to the excess MSE expressions for regularized estimators for moderate sample sizes and show its improvement on accuracy in numerical simulations.

Figures (6)

• Figure 1: Boxplots of the SE of $\hat{\bm\theta}^{\mathop{\mathrm{ML}}\limits}$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits})$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{Sy}}\limits})$ and $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{GCV}}\limits})$ for one specific system. Their sample MSE are $5.73\times 10^{-1}$, $8.49\times 10^{-1}$, $6.03\times 10^{-1}$ and $6.04\times 10^{-1}$, respectively.
• Figure 2: Sample $\Delta\text{MSE}(\cdot)$ ("sample $\Delta$MSE"), ${\mathop{\mathrm{XMSE}}\limits(\cdot)}/{N^2}$ ("XMSE"), and apx. ${\mathop{\mathrm{XMSE}}\limits(\cdot)}/{N^2}$ ("apx. XMSE") of $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits})$ in Panel (a) and of $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{Sy}}\limits})$ in Panel (b).
• Figure 3: Boxplots of average FIT of $\hat{\bm\theta}^{\mathop{\mathrm{ML}}\limits}$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits})$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{Sy}}\limits})$ and $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{GCV}}\limits})$ for systems with positive $\mathop{\mathrm{XMSE}}\limits(\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits}))$.
• Figure 4: Components of sample $\Delta\text{MSE}$ and apx. XMSE of $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits})$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{Sy}}\limits})$ and $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{GCV}}\limits})$ for systems with positive $\mathop{\mathrm{XMSE}}\limits(\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits}))$.
• Figure 5: Boxplots of average FIT of $\hat{\bm\theta}^{\mathop{\mathrm{ML}}\limits}$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{EB}}\limits})$, $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{Sy}}\limits})$ and $\hat{\bm\theta}^{\mathop{\mathrm{R}}\limits}(\hat{\eta}_{\mathop{\mathrm{GCV}}\limits})$ for systems with proper alignment between $\bm\theta_{0}$ and $\mathbf{P}(\eta)=\eta\mathbf{K}$.