Generative Learning of Continuous Data by Tensor Networks

TL;DR

This work extends tensor-network generative modeling to continuous data by introducing continuous-valued Born machines based on matrix product states. It proves universal approximation guarantees for both wavefunctions and probability densities using isometric feature maps, and adds a trainable compression layer to boost efficiency while preserving exact sampling capabilities. Empirically, the approach demonstrates strong density-estimation performance and sample quality across synthetic benchmarks (rotated hypercube, two moons) and real datasets (Iris, XY model), with competitive KL/JS divergences relative to established baselines and the ability to condition or sample from mixed data. Overall, the study provides both rigorous theoretical foundations and practical methods for applying quantum-inspired tensor networks to real-world continuous data problems.

Abstract

Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervised generative learning. While possessing many desirable features arising from their quantum-inspired nature, tensor network generative models have previously been largely restricted to binary or categorical data, limiting their utility in real-world modeling problems. We overcome this by introducing a new family of tensor network generative models for continuous data, which are capable of learning from distributions containing continuous random variables. We develop our method in the setting of matrix product states, first deriving a universal expressivity theorem proving the ability of this model family to approximate any reasonably smooth probability density function with arbitrary precision. We then benchmark the performance of this model on several synthetic and real-world datasets, finding that the model learns and generalizes well on distributions of continuous and discrete variables. We develop methods for modeling different data domains, and introduce a trainable compression layer which is found to increase model performance given limited memory or computational resources. Overall, our methods give important theoretical and empirical evidence of the efficacy of quantum-inspired methods for the rapidly growing field of generative learning.

Figures (15)

• Figure 1: Continuous-valued tensor network. The feature layer (magenta) is a tensor product of feature map operators $\zeta$ defined on each site, with the thicker purple edges denoting indices associated to continuous values. The feature layer is connected to the site indices of a discrete-valued tensor network (blue). The specific network above defines a function $\Phi(\mathbf{x})$, where $\mathbf{x} = (x_1, x_2, \ldots, x_{20})$.
• Figure 2: Continuous-valued MPS. (a) For the feature layer, the input at each site $x\in \mathbb{R}$ is a continuous variable, after mapping, it outputs a discrete vector of feature dimension $D$, which is directly connected to the tensor network layer (blue). $\chi$ and $D$ are hyper parameters controlling the dimensions of different bonds. (b) Graphical depiction of the continuous-valued function $\Phi$ defined in Eq. \ref{['eq:wave_cont']}.
• Figure 3: Graphical formulation of the isometric condition on the feature map $\zeta$, which is equivalent to the orthonormality requirement on feature functions expressed in Eq. \ref{['eq:orth_condition']}.
• Figure 4: Continuous-valued MPS canonical form. (a) The underlying discrete-valued MPS is required to be in canonical form with an orthogonality center (green dot tensor). When the feature maps additionally satisfy the orthonormality relations of Eq. \ref{['eq:orth_condition']}, then the continuous-valued MPS is said to be in continuous-valued MPS canonical form. (b-c) Graphical proof that the left (right) tensors constitute isometries from the left (right) bond spaces to the space of square-integrable functions acting on the left (right) set of continuous variables.
• Figure 5: Tensor network diagrams depicting how calculating the probabilities of a continuous-valued MPS BM can be considerably simplified. The MPS is taken to be in canonical form, with the orthonormal center (green dot tensor) on the leftmost site. (a) The marginal distribution $P(x_1,x_2,x_3)$ is given by integrating out the continuous variables $x_4, x_5$, which is trivial when the MPS is in continuous-valued canonical form. (b) The conditional probability $P(x_3 | x_1,x_2)$ used in the sampling process, which is facilitated by the computation of a $D \times D$ conditional density matrix $\sigma^{(3)}(x_1, x_2)$.

Theorems & Definitions (10)

• Theorem 1
• Theorem 2
• Theorem 3
• Theorem 3
• Lemma 1
• Theorem 3
• Theorem 4
• Lemma 2: bigoni2016spectral
• Theorem 3
• Lemma 3: Faà di Bruno