Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix

The Tien Mai

TL;DR

This paper develops a Bayesian framework for estimating high-dimensional sparse precision matrices within Gaussian graphical models, using a fully specified Horseshoe prior and a tempered (fractional) posterior to obtain theoretical guarantees when $p>n$. It proves posterior concentration rates at $\varepsilon_n = \frac{s\log(p/s)}{n}$ in the $\alpha$-Renyi divergence and derives Frobenius-norm concentration for the posterior mean, along with a high-probability bound. The authors extend the theory to model misspecification, presenting a general oracle inequality that bounds the discrepancy to the best sparse approximation, and provide simulation evidence validating robustness and the potential benefits of intermediate tempering levels $\alpha$. These results advance Bayesian sparse precision-matrix estimation by linking shrinkage priors, tempered posteriors, and high-dimensional concentration theory, with practical implications for GGMs in biology, economics, and network science.

Abstract

Precision matrices are crucial in many fields such as social networks, neuroscience, and economics, representing the edge structure of Gaussian graphical models (GGMs), where a zero in an off-diagonal position of the precision matrix indicates conditional independence between nodes. In high-dimensional settings where the dimension of the precision matrix \( p \) exceeds the sample size \( n \) and the matrix is sparse, methods like graphical Lasso, graphical SCAD, and CLIME are popular for estimating GGMs. While frequentist methods are well-studied, Bayesian approaches for (unstructured) sparse precision matrices are less explored. The graphical horseshoe estimate by \cite{li2019graphical}, applying the global-local horseshoe prior, shows superior empirical performance, but theoretical work for sparse precision matrix estimations using shrinkage priors is limited. This paper addresses these gaps by providing concentration results for the tempered posterior with the fully specified horseshoe prior in high-dimensional settings. Moreover, we also provide novel theoretical results for model misspecification, offering a general oracle inequality for the posterior. A concise set of simulations is performed to validate our theoretical findings.

Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix

TL;DR

This paper develops a Bayesian framework for estimating high-dimensional sparse precision matrices within Gaussian graphical models, using a fully specified Horseshoe prior and a tempered (fractional) posterior to obtain theoretical guarantees when . It proves posterior concentration rates at in the -Renyi divergence and derives Frobenius-norm concentration for the posterior mean, along with a high-probability bound. The authors extend the theory to model misspecification, presenting a general oracle inequality that bounds the discrepancy to the best sparse approximation, and provide simulation evidence validating robustness and the potential benefits of intermediate tempering levels . These results advance Bayesian sparse precision-matrix estimation by linking shrinkage priors, tempered posteriors, and high-dimensional concentration theory, with practical implications for GGMs in biology, economics, and network science.

Abstract

Precision matrices are crucial in many fields such as social networks, neuroscience, and economics, representing the edge structure of Gaussian graphical models (GGMs), where a zero in an off-diagonal position of the precision matrix indicates conditional independence between nodes. In high-dimensional settings where the dimension of the precision matrix exceeds the sample size and the matrix is sparse, methods like graphical Lasso, graphical SCAD, and CLIME are popular for estimating GGMs. While frequentist methods are well-studied, Bayesian approaches for (unstructured) sparse precision matrices are less explored. The graphical horseshoe estimate by \cite{li2019graphical}, applying the global-local horseshoe prior, shows superior empirical performance, but theoretical work for sparse precision matrix estimations using shrinkage priors is limited. This paper addresses these gaps by providing concentration results for the tempered posterior with the fully specified horseshoe prior in high-dimensional settings. Moreover, we also provide novel theoretical results for model misspecification, offering a general oracle inequality for the posterior. A concise set of simulations is performed to validate our theoretical findings.
Paper Structure (14 sections, 10 theorems, 44 equations, 1 table)

This paper contains 14 sections, 10 theorems, 44 equations, 1 table.

Table of Contents

  1. Introduction
  2. Problem and method
  3. Model
  4. Method
  5. Results
  6. Consistency of tempered posterior
  7. Concentration rates
  8. Result in the misspecified case
  9. Numerical examples
  10. Conclusion
  11. Proofs
  12. Proof of Theorem \ref{['thm_expectation']}
  13. Proof of Theorem \ref{['theorem_main']}
  14. Proof of Theorem \ref{['theorem_misspecified']}

Key Result

theorem 1

For any $\alpha \in (0,1)$, assume that Assumptions assum_true_precision_Spectrum, assum_true_precision, and assum_sparsity are satisfied. Given any $\Omega_0$ such that $\|\Omega_0\|_\infty \leq C_1$, then where $\varepsilon_n = \frac{s\log (p/s)}{n} ,$ and $C$ is a universal positive constant depending only on $\varepsilon_0, C_1$.

Theorems & Definitions (18)

  • theorem 1
  • Corollary 3.1
  • theorem 2
  • Remark 1
  • Corollary 3.2
  • Corollary 3.3
  • theorem 3
  • Corollary 3.4
  • proof
  • proof : Proof of Corollary \ref{['cor_expect_Frobenius']}
  • ...and 8 more