Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bayesian Prediction under Moment Conditioning

Nicholas G. Polson, Daniel Zantedeschi

TL;DR

The paper develops a finite, curvature-adaptive Bayesian framework for prediction under moment constraints, revealing a predictive-collapse phenomenon governed by the information-geometric Hessian $H^*$ and its smallest eigenvalue $\lambda_{\min}(H^*)$, yielding an explicit tilted Gaussian (finite de Finetti) representation around the information projection $P^*$. It unifies empirical likelihood, Bayesian empirical likelihood, ETEL/BETEL, GMM, and GEE within a common geometric picture, and derives a LAN-like limit with curvature-determined rates, including misspecification analyses via Berk's theorem and geometric tempering. A curvature-adaptive localization window $\mathcal{W}_n$ and a precise $O\big(m\sqrt{\log n/(n\lambda_{\min}(H^*))}\big)$ contraction bound quantify finite-sample predictive precision, while practical procedures for computing $\lambda_{\min}(H^*)$ provide identifiability diagnostics. The work offers a constructive bridge between partial information and full Bayesian prediction, with broad implications for constrained inference and robust uncertainty quantification in finite settings and guidance for extending to continuous spaces and online-learning contexts.

Abstract

Prediction is a central task of statistics and machine learning, yet many inferential settings provide only partial information, typically in the form of moment constraints or estimating equations. We develop a finite, fully Bayesian framework for propagating such partial information through predictive distributions. Building on de Finetti's representation theorem, we construct a curvature-adaptive version of exchangeable updating that operates directly under finite constraints, yielding an explicit discrete-Gaussian mixture that quantifies predictive uncertainty. The resulting finite-sample bounds depend on the smallest eigenvalue of the information-geometric Hessian, which measures the curvature and identification strength of the constraint manifold. This approach unifies empirical likelihood, Bayesian empirical likelihood, and generalized method-of-moments estimation within a common predictive geometry. On the operational side, it provides computable curvature-sensitive uncertainty bounds for constrained prediction; on the theoretical side, it recovers de Finetti's coherence, Doob's martingale convergence and local asymptotic normality as limiting cases of the same finite mechanism. Our framework thus offers a constructive bridge between partial information and full Bayesian prediction.

Bayesian Prediction under Moment Conditioning

TL;DR

The paper develops a finite, curvature-adaptive Bayesian framework for prediction under moment constraints, revealing a predictive-collapse phenomenon governed by the information-geometric Hessian and its smallest eigenvalue , yielding an explicit tilted Gaussian (finite de Finetti) representation around the information projection . It unifies empirical likelihood, Bayesian empirical likelihood, ETEL/BETEL, GMM, and GEE within a common geometric picture, and derives a LAN-like limit with curvature-determined rates, including misspecification analyses via Berk's theorem and geometric tempering. A curvature-adaptive localization window and a precise contraction bound quantify finite-sample predictive precision, while practical procedures for computing provide identifiability diagnostics. The work offers a constructive bridge between partial information and full Bayesian prediction, with broad implications for constrained inference and robust uncertainty quantification in finite settings and guidance for extending to continuous spaces and online-learning contexts.

Abstract

Prediction is a central task of statistics and machine learning, yet many inferential settings provide only partial information, typically in the form of moment constraints or estimating equations. We develop a finite, fully Bayesian framework for propagating such partial information through predictive distributions. Building on de Finetti's representation theorem, we construct a curvature-adaptive version of exchangeable updating that operates directly under finite constraints, yielding an explicit discrete-Gaussian mixture that quantifies predictive uncertainty. The resulting finite-sample bounds depend on the smallest eigenvalue of the information-geometric Hessian, which measures the curvature and identification strength of the constraint manifold. This approach unifies empirical likelihood, Bayesian empirical likelihood, and generalized method-of-moments estimation within a common predictive geometry. On the operational side, it provides computable curvature-sensitive uncertainty bounds for constrained prediction; on the theoretical side, it recovers de Finetti's coherence, Doob's martingale convergence and local asymptotic normality as limiting cases of the same finite mechanism. Our framework thus offers a constructive bridge between partial information and full Bayesian prediction.
Paper Structure (35 sections, 19 theorems, 58 equations)

This paper contains 35 sections, 19 theorems, 58 equations.

Table of Contents

  1. Introduction
  2. Connections with Prior Work
  3. Contributions
  4. Bayesian Prediction Under Constraints
  5. Constraint Structure and Geometry
  6. Predictive Representation
  7. Main Results
  8. Geometric mechanism
  9. Curvature-Adaptive Localization
  10. Tilted Finite Gaussian de Finetti Theorem
  11. Curvature and Predictive Contraction
  12. Computation of the Curvature
  13. Interpretation and Implications
  14. Asymptotic and Geometric Connections
  15. Practical Implications
  16. ...and 20 more sections

Key Result

Theorem 3.1

Let $\phi_{H^*}(v) = |H^*|^{1/2}(2\pi)^{-r/2} \exp\!(-\tfrac{1}{2}v^\top H^*v)$ denote the Gaussian density on $\mathsf{T}^*$. For measurable $A\subset\mathcal{X}^m$ with $m = O(n^\eta)$ for any fixed $\eta < 1/2$, where $P(v)=\Pi(P^*+n^{-1/2}v)$ is the local chart of $E$, and for a constant $C_{\mathrm{geo}}$ depending only on $(|\mathcal{X}|,d,p^*_{\min})$ and the local curvature near $P^*$.

Theorems & Definitions (28)

  • Theorem 3.1: Tilted Finite Gaussian de Finetti
  • Remark 3.2: Interpretation
  • Remark 3.3: Tilted finite de Finetti
  • Theorem 3.4: Predictive collapse bound
  • Remark 3.5: Interpretation
  • Corollary 4.1: Local asymptotic normality
  • Remark 4.2: Connection with Le Cam's theory
  • Remark 4.3: Coordinate invariance
  • Theorem 5.1: BETEL as conditional likelihood
  • Corollary 5.2: Posterior concentration
  • ...and 18 more