Generative Modeling via Hierarchical Tensor Sketching

TL;DR

The paper develops an optimization-free method to estimate high-dimensional probability densities from empirical data by constructing a hierarchical tensor-network (TT/MPS) representation. It leverages randomized sketching and a binary hierarchical decomposition to form a small system of tensor-core equations, achieving complexity that scales linearly with dimension as $O(N d\log d)$. A trimming strategy and carefully chosen sketch functions stabilize the method, and a Frobenius-norm error analysis shows the estimation error decays like $O( c^{\log d}/\sqrt{N} )$ under suitable assumptions, with high-probability guarantees. Numerical experiments on one- and two-dimensional Ising models demonstrate accurate density estimation and favorable rate behavior across varying ranks, temperatures, and lattice sizes, highlighting the method’s applicability to spatial random-field densities.

Abstract

We propose a hierarchical tensor-network approach for approximating high-dimensional probability density via empirical distribution. This leverages randomized singular value decomposition (SVD) techniques and involves solving linear equations for tensor cores in this tensor network. The complexity of the resulting algorithm scales linearly in the dimension of the high-dimensional density. An analysis of estimation error demonstrates the effectiveness of this method through several numerical experiments.

Figures (14)

• Figure 1: Tensor diagrams examples.
• Figure 2: Tensor diagram of core defining equations \ref{['eq:high d cdse']}. In order to go from left hand side to right hand side, we use the reshaping operation in Figure \ref{['fig:dg_demo']}(C) to reshape the dimension ($\bm{x}_{\mathcal{C}_{4k-3}^{(l+1)}},\bm{x}_{\mathcal{C}_{4k-2}^{(l+1)}}$) of $p_{2k-1}^{(l)}$ into $\bm{x}_{\mathcal{C}_{2k-1}^{(l)}}$ and reshape the dimension ($\bm{x}_{\mathcal{C}_{4k-1}^{(l+1)}},\bm{x}_{\mathcal{C}_{4k}^{(l+1)}}$) of $p_{2k}^{(l)}$ into $\bm{x}_{\mathcal{C}_{2k}^{(l)}}$.
• Figure 3: Tensor diagram of the hierarchical tensor-network representing an 8-dimensional density $p$.
• Figure 5: Tensor diagram of trimmed hierarchical tensor-network.
• Figure 6: Error for next neighbor one-dimensional Ising model with $d=16$. Blue, red, and black curves represent cases with $\beta=0.4$, $\beta=0.6$, and $\beta=0.8$, respectively. Dashed curve reflects a reference curve $O(N^{-1/2})$.

Theorems & Definitions (20)

• Theorem 2
• proof
• Theorem 3
• proof
• Proposition 4
• proof
• Theorem 6
• proof
• Remark 1
• Remark 2