Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximation of High-Dimensional Gibbs Distributions with Functional Hierarchical Tensors

Nan Sheng, Xun Tang, Haoxuan Chen, Lexing Ying

TL;DR

This work tackles the difficulty of approximating high-dimensional Gibbs distributions, whose normalization constant is intractable, by pairing ensemble-based annealed importance sampling (AIS) with functional hierarchical tensor (FHT) sketching to yield a normalized density representation $p(x)$. The main idea is to generate Gibbs samples via an AIS-driven, ensemble-based sampler that mitigates metastability, and then estimate the density through a low-rank FHT ansatz by matching moments with hierarchical sketching. The authors provide theoretical justification for using the FHT representation and demonstrate the method on complex Ginzburg-Landau models with hundreds of variables, showing effective capture of multimodal structure and metastability. This approach offers a scalable pathway to compute moments and marginals of high-dimensional Gibbs distributions, with potential applications in statistical mechanics, molecular dynamics, and high-dimensional inference.

Abstract

The numerical representation of high-dimensional Gibbs distributions is challenging due to the curse of dimensionality manifesting through the intractable normalization constant calculations. This work addresses this challenge by performing a particle-based high-dimensional parametric density estimation subroutine, and the input to the subroutine is Gibbs samples generated by leveraging advanced sampling techniques. Specifically, to generate Gibbs samples, we employ ensemble-based annealed importance sampling, a population-based approach for sampling multimodal distributions. These samples are then processed using functional hierarchical tensor sketching, a tensor-network-based density estimation method for high-dimensional distributions, to obtain the numerical representation of the Gibbs distribution. We successfully apply the proposed approach to complex Ginzburg-Landau models with hundreds of variables. In particular, we show that the approach proposed is successful at addressing the metastability issue under difficult numerical cases.

Approximation of High-Dimensional Gibbs Distributions with Functional Hierarchical Tensors

TL;DR

This work tackles the difficulty of approximating high-dimensional Gibbs distributions, whose normalization constant is intractable, by pairing ensemble-based annealed importance sampling (AIS) with functional hierarchical tensor (FHT) sketching to yield a normalized density representation . The main idea is to generate Gibbs samples via an AIS-driven, ensemble-based sampler that mitigates metastability, and then estimate the density through a low-rank FHT ansatz by matching moments with hierarchical sketching. The authors provide theoretical justification for using the FHT representation and demonstrate the method on complex Ginzburg-Landau models with hundreds of variables, showing effective capture of multimodal structure and metastability. This approach offers a scalable pathway to compute moments and marginals of high-dimensional Gibbs distributions, with potential applications in statistical mechanics, molecular dynamics, and high-dimensional inference.

Abstract

The numerical representation of high-dimensional Gibbs distributions is challenging due to the curse of dimensionality manifesting through the intractable normalization constant calculations. This work addresses this challenge by performing a particle-based high-dimensional parametric density estimation subroutine, and the input to the subroutine is Gibbs samples generated by leveraging advanced sampling techniques. Specifically, to generate Gibbs samples, we employ ensemble-based annealed importance sampling, a population-based approach for sampling multimodal distributions. These samples are then processed using functional hierarchical tensor sketching, a tensor-network-based density estimation method for high-dimensional distributions, to obtain the numerical representation of the Gibbs distribution. We successfully apply the proposed approach to complex Ginzburg-Landau models with hundreds of variables. In particular, we show that the approach proposed is successful at addressing the metastability issue under difficult numerical cases.

Paper Structure

This paper contains 31 sections, 19 equations, 6 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background and contribution
  3. Functional hierarchical tensor
  4. Density estimation through hierarchical sketching
  5. Annealed and ensemble samplers for moment estimation
  6. Contribution
  7. Related work
  8. Neural network-based methods for density estimation
  9. Tensor network-based methods for density estimation
  10. Ensemble-based annealed importance sampling
  11. Contents and notations
  12. Main formulation
  13. Ensemble-based AIS
  14. Annealed importance sampling
  15. Birth-death process
  16. ...and 16 more sections

Figures (6)

  • Figure 1: Illustrations of functional hierarchical tensor with $d = 8$tang2024solving.
  • Figure 1: 1D Ginzburg-Landau model $\lambda=0.1 h$. (a) Ratio defined in \ref{['eq:ratio']} as a function of temperature $\beta$. (b) Ratio as a function of time $t$. (c) Marginal distributions at $(x_{2}, x_{1})$.
  • Figure 2: 1D Ginzburg-Landau model $\lambda=h$. (a) Ratio as a function of temperature $\beta$. (b) Ratio as a function of time $t$. (c) Marginal distributions at $(x_{45}, x_{59})$.
  • Figure 3: 1D Asymmetric Ginzburg-Landau model $\lambda=0.5 h$ and $a=0.01$. (a) Ratio as a function of temperature $\beta$. (b) Ratio as a function of time $t$. (c) Marginal distributions at $(x_{15}, x_{59})$.
  • Figure 4: 2D Ginzburg-Landau model $\lambda=0.125 h$. (a) Ratio as a function of temperature $\beta$. (b) Ratio as a function of time $t$. (c) Marginal distributions at $(x_{(3,5)}, x_{(1,3)})$.
  • ...and 1 more figures