Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On A Necessary Condition For Posterior Inconsistency: New Insights From A Classic Counterexample

Nicola Bariletto, Stephen G. Walker

TL;DR

The paper investigates posterior consistency in Bayesian density estimation under KL support and seeks a substantive necessary condition for Hellinger inconsistency. By analyzing Barrons classical counterexample, it shows that inconsistency is driven by persistent posterior mass on densities with exponentially large likelihood ratios in a narrow range, which in turn requires an unrealistic alignment between the prior, likelihood, and the unknown true distribution. The authors introduce $(\alpha,\beta)$- and $\gamma$-inconsistency and prove that, in a broad class of $\beta$-bounded models, such inconsistency can only arise under highly contrived prior specifications that encode precise knowledge of the data-generating process. Consequently, they argue that Barrons inconsistency is a pathology of construction rather than a natural phenomenon, reinforcing the practical robustness of Bayesian density estimation in realistic settings.

Abstract

The consistency of posterior distributions in density estimation is at the core of Bayesian statistical theory. Classical work established sufficient conditions, typically combining KL support with complexity bounds on sieves of high prior mass, to guarantee consistency with respect to the Hellinger distance. Yet no systematic theory explains a widely held belief: under KL support, Hellinger consistency is exceptionally hard to violate. This suggests that existing sufficient conditions, while useful in practice, may overlook some key aspects of posterior behavior. We address this gap by directly investigating what must fail for inconsistency to arise, aiming to identify a substantive necessary condition for Hellinger inconsistency. Our starting point is Andrew Barron's classical counterexample, the only known violation of Hellinger consistency under KL support, which relies on a contrived family of oscillatory densities and a prior with atoms. We show that, within a broad class of models including Barron's, inconsistency requires persistent posterior concentration on densities with exponentially high likelihood ratios. In turn, such behavior demands a prior encoding implausibly precise knowledge of the true, yet unknown data-generating distribution, making inconsistency essentially unattainable in any realistic inference problem. Our results confirm the long-standing intuition that posterior inconsistency in density estimation is not a natural phenomenon, but rather an artifact of pathological prior constructions.

On A Necessary Condition For Posterior Inconsistency: New Insights From A Classic Counterexample

TL;DR

The paper investigates posterior consistency in Bayesian density estimation under KL support and seeks a substantive necessary condition for Hellinger inconsistency. By analyzing Barrons classical counterexample, it shows that inconsistency is driven by persistent posterior mass on densities with exponentially large likelihood ratios in a narrow range, which in turn requires an unrealistic alignment between the prior, likelihood, and the unknown true distribution. The authors introduce - and -inconsistency and prove that, in a broad class of -bounded models, such inconsistency can only arise under highly contrived prior specifications that encode precise knowledge of the data-generating process. Consequently, they argue that Barrons inconsistency is a pathology of construction rather than a natural phenomenon, reinforcing the practical robustness of Bayesian density estimation in realistic settings.

Abstract

The consistency of posterior distributions in density estimation is at the core of Bayesian statistical theory. Classical work established sufficient conditions, typically combining KL support with complexity bounds on sieves of high prior mass, to guarantee consistency with respect to the Hellinger distance. Yet no systematic theory explains a widely held belief: under KL support, Hellinger consistency is exceptionally hard to violate. This suggests that existing sufficient conditions, while useful in practice, may overlook some key aspects of posterior behavior. We address this gap by directly investigating what must fail for inconsistency to arise, aiming to identify a substantive necessary condition for Hellinger inconsistency. Our starting point is Andrew Barron's classical counterexample, the only known violation of Hellinger consistency under KL support, which relies on a contrived family of oscillatory densities and a prior with atoms. We show that, within a broad class of models including Barron's, inconsistency requires persistent posterior concentration on densities with exponentially high likelihood ratios. In turn, such behavior demands a prior encoding implausibly precise knowledge of the true, yet unknown data-generating distribution, making inconsistency essentially unattainable in any realistic inference problem. Our results confirm the long-standing intuition that posterior inconsistency in density estimation is not a natural phenomenon, but rather an artifact of pathological prior constructions.
Paper Structure (7 sections, 6 theorems, 23 equations)

This paper contains 7 sections, 6 theorems, 23 equations.

Table of Contents

  1. Introduction
  2. Formal setup and preliminary results
  3. Barron's counterexample
  4. Inconsistency
  5. A necessary condition for Hellinger inconsistency
  6. Conclusion
  7. Future work

Key Result

Lemma 1

Assume that $f_\star$ lies in the KL support of $\Pi$. Then, for any weak neighborhood $U$ of $F_\star$, $\lim_{n\to\infty} \Pi(U^c \mid X_{1:n}) = 0$ almost surely. Moreover, $\int_\mathbb{F} R_n(f)\,\Pi(\mathrm df) \geq e^{-\tau n}$ ultimately almost surely, for all $\tau > 0$.

Theorems & Definitions (15)

  • Definition 1
  • Lemma 1
  • Lemma 2
  • proof
  • Definition 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Definition 3
  • ...and 5 more