Representation Theorem for Matrix Product States
Erdong Guo, David Draper
TL;DR
This work analyzes the representation capacity of Matrix Product States (MPS) from both combinatorial and analytic perspectives. It proves that MPS can exactly realize arbitrary boolean functions and, with a scale-invariant sigmoidal activation, that their function class is dense in $C^{0}(I^{n})$, enabling universal approximation of continuous mappings. A key insight is the equivalence between activated MPS and one-hidden-layer neural networks equipped with kernel functions, which naturally induce nonlinear input interactions; explicit constructions illustrate polynomial-like kernels arising from MPS. The paper also shows that infinite-width MPS converge to Gaussian Processes, connecting tensor-network models to well-studied probabilistic priors. These results synthesize tensor networks and kernelized neural networks, offering a principled pathway to design MPS-based models with rich representational power for Boolean and continuous tasks.
Abstract
In this work, we investigate the universal representation capacity of the Matrix Product States (MPS) from the perspective of boolean functions and continuous functions. We show that MPS can accurately realize arbitrary boolean functions by providing a construction method of the corresponding MPS structure for an arbitrarily given boolean gate. Moreover, we prove that the function space of MPS with the scale-invariant sigmoidal activation is dense in the space of continuous functions defined on a compact subspace of the $n$-dimensional real coordinate space $\mathbb{R^{n}}$. We study the relation between MPS and neural networks and show that the MPS with a scale-invariant sigmoidal function is equivalent to a one-hidden-layer neural network equipped with a kernel function. We construct the equivalent neural networks for several specific MPS models and show that non-linear kernels such as the polynomial kernel which introduces the couplings between different components of the input into the model appear naturally in the equivalent neural networks. At last, we discuss the realization of the Gaussian Process (GP) with infinitely wide MPS by studying their equivalent neural networks.