Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Posterior Consistency in Parametric Models via a Tighter Notion of Identifiability

Nicola Bariletto, Bernardo Flores, Stephen G. Walker

TL;DR

The paper challenges the view that parametric posterior consistency is fully resolved by classical regularity conditions, arguing that identifiability—and in particular sequential identifiability in an augmented, weak-limit-inclusive parameter space—is the essential driver. Building on Schwartz’s weak consistency, the authors show that, under KL support, posterior consistency at the true parameter follows from sequential identifiability, while inconsistency would require highly contrived oscillations of the densities around the true density. They illustrate the theory with an oscillatory cosine-based example where classical conditions fail and the MLE is inconsistent, yet posterior consistency can be achieved for most parameter values, and they discuss how mild priors can restore consistency at challenging points like the uniform density. The work highlights a fundamental difference between parametric and nonparametric models, clarifies the distinct mechanisms behind Bayesian and frequentist inconsistency, and provides practical guidance for constructing priors and models that ensure reliable parametric inference.

Abstract

We study Bayesian posterior consistency in parametric density models with proper priors, challenging the perception that the problem is settled. Classical results established consistency via MLE convergence under regularity and identifiability assumptions, with the latter taken for granted and rarely examined. We refocus attention on identifiability, showing that inconsistency arises only when the true distribution coincides with a weak limit of model densities in a way that violates identifiability. While such failures occur naturally in nonparametric settings, they are implausible and effectively self-inflicted in parametric models. Our analysis shows that classical regularity conditions are unnecessary: a mild strengthening of identifiability suffices to ensure consistency in parametric models, even when the MLE is inconsistent. We also demonstrate that parametric inconsistency requires carefully engineered, oscillatory model features aligned with the true distribution, which is unlikely to occur without adversarial design. Our findings also clarify the distinct mechanisms behind Bayesian and frequentist inconsistency and advocate for separate theoretical treatments.

Posterior Consistency in Parametric Models via a Tighter Notion of Identifiability

TL;DR

The paper challenges the view that parametric posterior consistency is fully resolved by classical regularity conditions, arguing that identifiability—and in particular sequential identifiability in an augmented, weak-limit-inclusive parameter space—is the essential driver. Building on Schwartz’s weak consistency, the authors show that, under KL support, posterior consistency at the true parameter follows from sequential identifiability, while inconsistency would require highly contrived oscillations of the densities around the true density. They illustrate the theory with an oscillatory cosine-based example where classical conditions fail and the MLE is inconsistent, yet posterior consistency can be achieved for most parameter values, and they discuss how mild priors can restore consistency at challenging points like the uniform density. The work highlights a fundamental difference between parametric and nonparametric models, clarifies the distinct mechanisms behind Bayesian and frequentist inconsistency, and provides practical guidance for constructing priors and models that ensure reliable parametric inference.

Abstract

We study Bayesian posterior consistency in parametric density models with proper priors, challenging the perception that the problem is settled. Classical results established consistency via MLE convergence under regularity and identifiability assumptions, with the latter taken for granted and rarely examined. We refocus attention on identifiability, showing that inconsistency arises only when the true distribution coincides with a weak limit of model densities in a way that violates identifiability. While such failures occur naturally in nonparametric settings, they are implausible and effectively self-inflicted in parametric models. Our analysis shows that classical regularity conditions are unnecessary: a mild strengthening of identifiability suffices to ensure consistency in parametric models, even when the MLE is inconsistent. We also demonstrate that parametric inconsistency requires carefully engineered, oscillatory model features aligned with the true distribution, which is unlikely to occur without adversarial design. Our findings also clarify the distinct mechanisms behind Bayesian and frequentist inconsistency and advocate for separate theoretical treatments.

Paper Structure

This paper contains 16 sections, 12 theorems, 106 equations, 2 figures.

Table of Contents

  1. Introduction
  2. A new understanding of parametric consistency
  3. Layout of the paper.
  4. Notation and basic definitions
  5. Classical conditions for parametric posterior consistency
  6. An alternative MLE-based strategy
  7. Sequential identifiability, oscillations and posterior consistency
  8. The role of oscillations
  9. Example 1: exponential families
  10. Example 2: uniform distribution
  11. Example 3: finite mixture models
  12. An illustrative model
  13. Some remarks on the illustrative model
  14. Failure of classical conditions
  15. Consistency at the uniform density
  16. ...and 1 more sections

Key Result

Theorem 1

Let $\theta_\star\in\textnormal{KLS}(\Pi)$. Then the posterior is weakly consistent at $\theta_\star$, that is, a.s.-$F_{\theta_\star}^\infty$ for all $\varepsilon > 0$.

Figures (2)

  • Figure 1: The initial segment of a sequence $f_1, f_2, \dots$ of indefinitely oscillating densities (solid gray, top row), whose corresponding sequence of CDFs (solid gray, bottom row) converges to a proper limiting distribution (dashed black). Such convergence is possible because the density sequence exhibits increasingly frequent oscillations around the density associated with the limiting CDF.
  • Figure 2: Density functions (solid gray, top row) and CDFs (solid gray, bottom row) in the illustrative parametric model, shown for increasing values of the parameter $\theta$. As $\theta$ grows, the density becomes increasingly oscillatory while the CDF converges to $F_0$ (dashed black).

Theorems & Definitions (18)

  • Definition 1
  • Definition 2
  • Theorem 1: Schwartz
  • Definition 3
  • Theorem 2
  • Theorem 3
  • Proposition 1
  • Corollary 1
  • Proposition 2
  • Theorem 4
  • ...and 8 more