Hierarchic Flows to Estimate and Sample High-dimensional Probabilities

TL;DR

This work addresses the challenge of modeling high-dimensional, non-Gaussian fields by introducing hierarchic probability flows that operate across wavelet scales. By formulating an inverse renormalization group in a wavelet basis, the authors decompose a high-dimensional Gibbs energy into a cascade of low-dimensional conditional models, enabling efficient estimation via score matching and sampling via Langevin-type dynamics. They show that renormalizing wavelet coefficients can stabilize log-Sobolev constants, mitigating critical slowing down in challenging cases such as the $\varphi^4$ model at criticality, and demonstrate practical generation of 2D turbulence and dark matter density fields using wavelet scattering energies. The framework provides a principled balance between spatial and spectral localization, supports hierarchical potentials that grow with scale, and yields tractable parameter counts ($O(\log^3 d)$ or similar) for capturing multiscale interactions. Overall, the method offers a scalable, interpretable route to learn and sample complex multiscale processes with strong non-Gaussian structure, with potential extensions to higher dimensions and dynamic settings.

Abstract

Finding low-dimensional interpretable models of complex physical fields such as turbulence remains an open question, 80 years after the pioneer work of Kolmogorov. Estimating high-dimensional probability distributions from data samples suffers from an optimization and an approximation curse of dimensionality. It may be avoided by following a hierarchic probability flow from coarse to fine scales. This inverse renormalization group is defined by conditional probabilities across scales, renormalized in a wavelet basis. For a $\vvarphi^4$ scalar potential, sampling these hierarchic models avoids the critical slowing down at the phase transition. In a well chosen wavelet basis, conditional probabilities can be captured with low dimensional parametric models, because interactions between wavelet coefficients are local in space and scales. An outstanding issue is also to approximate non-Gaussian fields having long-range interactions in space and across scales. We introduce low-dimensional models of wavelet conditional probabilities with the scattering covariance. It is calculated with a second wavelet transform, which defines interactions over two hierarchies of scales. We estimate and sample these wavelet scattering models to generate 2D vorticity fields of turbulence, and images of dark matter densities.

Figures (15)

• Figure 1: The renormalization group (illustrated in blue) computes the probability distributions $p_j (\varphi_j)$ of images $\varphi_j$ at progressively larger scales $2^j$, with marginal integrations of high-frequency degrees of freedom, up to a maximum scale $2^J$. A hierarchic flow (illustrated in red) is a top-down inverse renormalization group which recovers $p$ from $p_J$ by estimating each transition probability $\bar{p}_j$ from $p_j$ to $p_{j-1}$marchand_wavelet_2022. Difficulties arise when $\bar{p}_j$ is non-local, as in turbulent flows.
• Figure 2: A fast wavelet transform iteratively decomposes an image approximation $\varphi_{j-1}$ into a coarser approximation $\varphi_{j}$ with a sub-sampled low-pass filtering $G$, and $3$ wavelet coefficient images $\bar{\varphi}_{j}$. They are calculated by $\bar{G}$ with subsampled convolutions with $3$ band-pass filters along different orientations. A finer scale image $\varphi_{j-1}$ is reconstructed from $(\varphi_j , \bar{\varphi}_j)$ with the inverse operator $(H,\bar{H})$.
• Figure 3: (a): Frequency supports of Fourier transforms of two-dimensional wavelets $\hat{\psi}_{j,k}(\omega_1,\omega_2)$ for $1 \leq k \leq 3$ over $3$ scales $2^j$. (b): Frequency subdivisions of wavelet packets over $a_j$ levels at each scale $2^j$, with $a_1 = 2$, $a_2 = 1$ and $a_3 = 0$.
• Figure 4: Top Row : Realisations of $\varphi^4$ fields at temperature $1/\beta$, and system size $d=128^2$. (a): For $\beta =0.5 < \beta_c$ , the system is disordered with short range correlations. (b): At the phase transition, $\beta = 0.68\approx \beta_c$, the field is self-similar, with long range correlations. (c): For $\beta = 0.76 > \beta_c$, the system is in a ferromagnetic phase, with a non-zero mean (here positive). Bottom row: samples generated with a hierarchic factorization in a Haar wavelet basis with the same $\beta$ in (a,b,c). (d) : The graph shows the covariance eigenvalues (power spectrum) in these $3$ cases, as a function of the two-dimensional frequency radius $|\omega| = (|{\omega}_1|^2 + |\omega_2|^2)^{1/2}$. For $\beta = \beta_c$, eigenvalues have a power-law decay and develop a singularity at low frequencies, which correspond to long-range correlations. We superimposed in dashed line the covariance eigenvalues of a hierarchic model estimated by score matching. For visualization, the spectrum at different temperatures are multiplied by a constant which aligns their minimum eigenvalue.
• Figure 5: (a): We consider images $\varphi_0$ of the $\varphi^4$ model of dimension $d=128^2$, for a critical $\beta = \beta_c$. The approximations $\varphi_1$ are calculated with a Symlet-4 filter at the finest scale $2^1$. The graph shows the distributions of eigenvalues of the Hessian $\nabla^2_{\varphi_{0}} {U}_{0}$ in blue, and of $\nabla^2_{\bar{\varphi}_1} \bar{{U}}_1= \nabla^2_{\bar{\varphi}_1} {U}_0$ (without renormalization) in orange. The most negative eigenvalues of $\nabla^2_{\varphi_{0}} {U}_{0}$ correspond to low frequency eigenvectors. They do not appear in $\nabla^2_{\bar{\varphi}_1} \bar{{U}}_1$. (b): Distributions of eigenvalues of Hessians $\nabla^2_{\bar{\varphi}_j} \bar{{U}}_{\theta_j}$ at all scales $2^j \leq 2^J$, for $d=32^2$, with $J=3$, for samples of hierarchical models of $\varphi^4$ at phase transition. They are computed for Haar (blue), Symlet-4 (orange) and Shannon wavelets (green). Eigenvalues are more concentrated when the wavelet has a better frequency localization.

Theorems & Definitions (8)

• Definition 2.1
• Proposition 2.1
• Proposition 3.1
• Definition 3.1
• Proposition 3.2
• proof
• Theorem 4.1
• Theorem 5.1