Sharp detection of low-dimensional structure in probability measures via dimensional logarithmic Sobolev inequalities

TL;DR

The paper develops a dimension-reduction framework for high-dimensional probability measures by approximating $\pi$ as a low-dimensional perturbation of a reference $\mu$ along $r$ linear features $U_r$. It establishes certifiable bounds for KL divergence via dimensional logarithmic Sobolev inequalities, showing that the minimizers are given by leading eigenvectors of the relative Fisher information $H(\pi\|\mu)$ and that dimensional refinements yield substantially tighter certificates than standard LSI, with exact results in Gaussian settings. The methodology extends to the squared Hellinger distance via the dimensional Poincaré inequality and to data-free Bayesian settings, including applications with generative priors, demonstrating that only a modest number of features (often far fewer than $d$) capture most of the informative structure. These results provide practical, provable guidance for feature discovery in Bayesian inverse problems and generative-model priors, enabling efficient, certifiable dimension reduction for posterior sampling and inference.

Abstract

Identifying low-dimensional structure in high-dimensional probability measures is an essential pre-processing step for efficient sampling. We introduce a method for identifying and approximating a target measure $π$ as a perturbation of a given reference measure $μ$ along a few significant directions of $\mathbb{R}^{d}$. The reference measure can be a Gaussian or a nonlinear transformation of a Gaussian, as commonly arising in generative modeling. Our method extends prior work on minimizing majorizations of the Kullback--Leibler divergence to identify optimal approximations within this class of measures. Our main contribution unveils a connection between the \emph{dimensional} logarithmic Sobolev inequality (LSI) and approximations with this ansatz. Specifically, when the target and reference are both Gaussian, we show that minimizing the dimensional LSI is equivalent to minimizing the KL divergence restricted to this ansatz. For general non-Gaussian measures, the dimensional LSI produces majorants that uniformly improve on previous majorants for gradient-based dimension reduction. We further demonstrate the applicability of this analysis to the squared Hellinger distance, where analogous reasoning shows that the dimensional Poincaré inequality offers improved bounds.

Figures (7)

• Figure 1: Comparison of approximation error certificates for a Gaussian target measure of size $d = n_y = 50$. The globally optimal features $U_r^\mathrm{opt}$ are obtained by minimizing \ref{['eq:dimCDR_up']}, and their achieved KL error is shown in black. The certificate obtained from the LSI majorization \ref{['eq:CDRbound']} is shown in gray. The minimizers of the empirically estimated dimensional LSI majorant \ref{['eq:dimCDR_up']} is shown by the coloured curves, where solid lines correspond to testing error, and dashed lines correspond to training error.
• Figure 2: (Left) Probability density of target measure. (Centre) Approximate measure using feature $U_1^*$ that minimizes the majorant in \ref{['eq:CDRbound']}. (Right) Approximate measure using feature $U_1^\downarrow$ that minimizes the majorant in \ref{['eq:dimCDRbound']}.
• Figure 3: (Rosenbrock) KL divergence $\mathop{\mathrm{D_\mathrm{KL}}}\nolimits(\pi|| \widetilde{\pi}^{\mathrm{KL}}(U_1(\theta)))$ with $U_1(\theta)=(\cos(\theta),\sin(\theta))$ for $\theta\in[0^\circ,180^\circ)$, shown by the solid blue line, and its bounds obtained either by the LSI of Theorem \ref{['thm:CDR']}, depicted by the gray shaded region, or by the dimensional LSI of Theorem \ref{['thm:dimCDR']}, depicted by the blue shaded region.
• Figure 4: (Left) Probability density of target measure. (Centre) Optimal Gaussian prior with mean $m(\pi)$ and covariance $C(\pi)$. (Right) Approximate measure with feature $U_r$ using optimal Gaussian prior.
• Figure 5: (Rosenbrock) KL divergence $\mathop{\mathrm{D_\mathrm{KL}}}\nolimits(\pi|| \widetilde{\pi}^{\mathrm{KL}}(U_1(\theta)))$ with $U_1(\theta)=(\cos(\theta),\sin(\theta))$ for $\theta\in[0^\circ,180^\circ)$ (red line) with the optimal Gaussian reference measure $\mu = \mathcal{N}(m(\pi), C(\pi))$ and its bounds obtained by the dimensional LSI of Theorem \ref{['thm:dimCDR_tilted']} shown by the shaded red region. We include for comparison the true approximation error (blue line) and dimensional LSI bounds (blue shade) in Theorem \ref{['thm:dimCDR']} when constructing an approximation using the standard reference measure $\mu = \mathcal{N}(0,1)$.

Theorems & Definitions (34)

• Proposition 1: Gaussian LSI
• Theorem 2.1
• Proposition 2: Dimensional Gaussian LSI
• Remark 2.2
• Remark 2.3
• Theorem 2.4
• Remark 2.5
• Proposition 3
• Conjecture 2.6
• Proposition 4