Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The minimax optimal convergence rate of posterior density in the weighted orthogonal polynomials

Yiqi Luo, Xue Luo

Abstract

We investigate Bayesian nonparametric density estimation via orthogonal polynomial expansions in weighted Sobolev spaces. A core challenge is establishing minimax optimal posterior convergence rates, especially for densities on unbounded domains without a strictly positive lower bound. For densities bounded away from zero, we give sufficient conditions under which the framework of \cite{shen2001} applies directly. For densities lacking a positive lower bound, the equivalence between Hellinger and weighted $L_2$-norm distance fails, invalidating the original theory. We propose a novel shifting method that lifts the true density $g_0$ to a sequence of proxy densities $g_{0,n}$. We prove a modified convergence theorem applicable to these shifted densities, preserving the optimal rate. We also construct a Gaussian sieve prior that achieves the minimax rate $\varepsilon_n=n^{-p/(2p+1)}$ for any integer $p\geq1$. Numerical results confirm that our estimator approximates the true density well and validates the theoretical convergence rate.

The minimax optimal convergence rate of posterior density in the weighted orthogonal polynomials

Abstract

We investigate Bayesian nonparametric density estimation via orthogonal polynomial expansions in weighted Sobolev spaces. A core challenge is establishing minimax optimal posterior convergence rates, especially for densities on unbounded domains without a strictly positive lower bound. For densities bounded away from zero, we give sufficient conditions under which the framework of \cite{shen2001} applies directly. For densities lacking a positive lower bound, the equivalence between Hellinger and weighted -norm distance fails, invalidating the original theory. We propose a novel shifting method that lifts the true density to a sequence of proxy densities . We prove a modified convergence theorem applicable to these shifted densities, preserving the optimal rate. We also construct a Gaussian sieve prior that achieves the minimax rate for any integer . Numerical results confirm that our estimator approximates the true density well and validates the theoretical convergence rate.
Paper Structure (10 sections, 9 theorems, 76 equations, 2 figures)

This paper contains 10 sections, 9 theorems, 76 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Notations and Preliminaries
  3. Convergence Theorems
  4. Convergence theorem for the strictly positive lower bound case
  5. Convergence theorem for the case without strictly positive lower bound
  6. Construction of Sieve Priors
  7. Numerical experiments
  8. Experiment 1: The Bayesian estimator
  9. Experiment 2: the Hellinger distance v.s. the sample size
  10. Discussion

Key Result

Lemma 2.1

For any if where $\tilde{\gamma}_j$ is any constant greater than or equal to for $j\geq1$, with $a_{ij}^{(l)}=\frac{1}{\gamma_i}\int_I q_j^{(l)}(x)q_i(x)w(x)\,dx$ being the coefficients in the expansion for $0\leq l\leq\min(j,p)$.

Figures (2)

  • Figure 1: True density $g_0(x)$ (red dashed) and the posterior mean density of $\hat{g}(\cdot|\mathbf{Y}^n)$ (blue) using the generalized Legendre polynomials and the trigonometric basis, and their $95\%$ posterior confidence bands.
  • Figure 2: The $m=100$ Hellinger distances with different sample sizes.

Theorems & Definitions (24)

  • Lemma 2.1
  • Remark 2.2
  • Definition 2.3: posterior convergence rate
  • Definition 2.4: Hellinger distance and KL neighborhood
  • Definition 2.5: bracketing entropy, Section 2.7, b20
  • Theorem 2.6: Theorem 4, shen2001
  • Remark 2.7
  • Remark 2.8
  • Remark 2.9
  • Theorem 3.1
  • ...and 14 more