Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

$L^1$ Estimation: On the Optimality of Linear Estimators

Leighton P. Barnes, Alex Dytso, Jingbo Liu, H. Vincent Poor

TL;DR

This work characterizes when the Bayes estimator under the $L^1$ loss is linear in the observation for the Gaussian-noise model. Using a convolution formulation and Fourier analysis on tempered distributions, it proves that the conditional median is linear if and only if $a\in[0,1)$ and $X$ is Gaussian with variance $\frac{a}{1-a}$; positivity of the associated measure is crucial to the analysis. The study reveals a phase transition across $L^p$ losses, with Gaussian priors uniquely inducing linearity for $p\in[1,2]$ and infinitely many priors enabling linearity for $p>2$, and it discusses extensions to Poisson-like noise, exponential-family models, and higher-dimensional settings, highlighting practical implications for prior design and estimator benchmarks in Bayesian estimation. Overall, the results complete the understanding of when linear Bayesian estimators arise under $L^1$ fidelity and connect to broader themes in convolution equations and exponential-family conjugacy.

Abstract

Consider the problem of estimating a random variable $X$ from noisy observations $Y = X+ Z$, where $Z$ is standard normal, under the $L^1$ fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on $X$ that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution $P_{X|Y=y}$ is symmetric for all $y$, then $X$ must follow a Gaussian distribution. Additionally, we consider other $L^p$ losses and observe the following phenomenon: for $p \in [1,2]$, Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for $p \in (2,\infty)$, infinitely many prior distributions on $X$ can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.

$L^1$ Estimation: On the Optimality of Linear Estimators

TL;DR

This work characterizes when the Bayes estimator under the loss is linear in the observation for the Gaussian-noise model. Using a convolution formulation and Fourier analysis on tempered distributions, it proves that the conditional median is linear if and only if and is Gaussian with variance ; positivity of the associated measure is crucial to the analysis. The study reveals a phase transition across losses, with Gaussian priors uniquely inducing linearity for and infinitely many priors enabling linearity for , and it discusses extensions to Poisson-like noise, exponential-family models, and higher-dimensional settings, highlighting practical implications for prior design and estimator benchmarks in Bayesian estimation. Overall, the results complete the understanding of when linear Bayesian estimators arise under fidelity and connect to broader themes in convolution equations and exponential-family conjugacy.

Abstract

Consider the problem of estimating a random variable from noisy observations , where is standard normal, under the fidelity criterion. It is well known that the optimal Bayesian estimator in this setting is the conditional median. This work shows that the only prior distribution on that induces linearity in the conditional median is Gaussian. Along the way, several other results are presented. In particular, it is demonstrated that if the conditional distribution is symmetric for all , then must follow a Gaussian distribution. Additionally, we consider other losses and observe the following phenomenon: for , Gaussian is the only prior distribution that induces a linear optimal Bayesian estimator, and for , infinitely many prior distributions on can induce linearity. Finally, extensions are provided to encompass noise models leading to conditional distributions from certain exponential families.
Paper Structure (22 sections, 17 theorems, 111 equations, 2 figures)

This paper contains 22 sections, 17 theorems, 111 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Outline and Contributions
  3. The Problem Setup and Some Preliminary Observations
  4. The Problem Setup
  5. On the Admissible Values of $a$
  6. Gaussian $X$ is a Solution
  7. An Equivalent Condition via Convolution
  8. Operator Theory Perspective and Why the Positivity Assumption Is Important
  9. Main Results
  10. Discussion, Extensions and Future Directions
  11. A Near Miss for Poisson Noise
  12. On Other $L^p$ Losses
  13. Extension to the Natural Exponential Family
  14. Extension to Higher Dimensions
  15. Proof of Theorem \ref{['thm:passing_gabbor_wavelet']}
  16. ...and 7 more sections

Key Result

Theorem 1

Let $p \ge 1$. Then, In other words, the admissible values of $a$ lie in $[0,1]$.

Figures (2)

  • Figure 1: Poisson Noise Case: Conditional mean vs. conditional median under a gamma prior with $(\alpha,\beta)= (1,1)$.
  • Figure 2: Example of probability densities in \ref{['eq:lp_densities']} for $p=4$ and $\rho=1$.

Theorems & Definitions (35)

  • Theorem 1
  • proof
  • Proposition 2
  • proof
  • Theorem 3
  • proof
  • Proposition 4
  • proof
  • Proposition 5
  • proof
  • ...and 25 more