Multivariate Priors and the Linearity of Optimal Bayesian Estimators under Gaussian Noise

TL;DR

This work characterizes when the Bayes estimator for estimating a random vector $X$ from Gaussian-noise observations $Y = X + Z$ under $L^p$ loss is linear. It shows a sharp dichotomy: for $1 \le p \le 2$, the optimal estimator is linear if and only if $X$ is multivariate Gaussian with covariance $\mathrm{Cov}(X) = (I-A)^{-1}A$ for some $0 \prec A \prec I$, while for $p>2$ there exist infinitely many priors that yield linear estimators. The analysis leverages a convolution identity, an orthogonality-like condition, and tempered-distribution/Fourier techniques to fully characterize the prior required for linearity in the $1 \le p \le 2$ regime and to construct nontrivial priors when $p>2$. The results clarify how the prior shape governs estimator linearity in Gaussian-noise channels and inform priors design for high-dimensional Bayesian linearity properties.

Abstract

Consider the task of estimating a random vector $X$ from noisy observations $Y = X + Z$, where $Z$ is a standard normal vector, under the $L^p$ fidelity criterion. This work establishes that, for $1 \leq p \leq 2$, the optimal Bayesian estimator is linear and positive definite if and only if the prior distribution on $X$ is a (non-degenerate) multivariate Gaussian. Furthermore, for $p > 2$, it is demonstrated that there are infinitely many priors that can induce such an estimator.

Figures (1)

• Figure 1: Example of probability densities in \ref{['eq:lp_densities']} for $p=4$ and $\rho=1$.

Theorems & Definitions (13)

• Theorem 1
• Proposition 2
• proof
• Proposition 3
• proof
• Lemma 4
• Lemma 5
• proof
• Lemma 6
• proof