Tensor network to learn the wavefunction of data

TL;DR

This work introduces a tensor network architecture which simultaneously accomplishes both classification (discrimination) and sampling tasks and characterize the full set by calculating its entanglement entropy.

Abstract

How many different ways are there to handwrite digit 3? To quantify this question imagine extending a dataset of handwritten digits MNIST by sampling additional images until they start repeating. We call the collection of all resulting images of digit 3 the "full set." To study the properties of the full set we introduce a tensor network architecture which simultaneously accomplishes both classification (discrimination) and sampling tasks. Qualitatively, our trained network represents the indicator function of the full set. It therefore can be used to characterize the data itself. We illustrate that by studying the full sets associated with the digits of MNIST. Using quantum mechanical interpretation of our network we characterize the full set by calculating its entanglement entropy. We also study its geometric properties such as mean Hamming distance, effective dimension, and size. The latter answers the question above -- the total number of black and white threes written MNIST style is $2^{72}$.

Figures (10)

• Figure 1: On both panels $I$ is the set of all $2^{784}$ images, $F$ is the full set of images of digit $i$. Let panel: ideal discriminator $\Psi$ recognizes all images of digit $i$, but may also recognize as $i$ images of other digits or noise. This means $|\Psi(x)^2|\geq \epsilon$ for all $x\in F$, as well as for some set $x\in R$ depicted in red. Right panel: all images sampled by an ideal sampler $\Psi$ are images of $i$. This means $\Psi$ has the support on the subset of the full set $S\subset F$, while $|\Psi(x)|^2 \approx 0$ for $x\notin S$.
• Figure 2: Left panel: quality of sampling by $\Psi_3$ with the bond dimension $D = 100$ during the training process. Quality is assessed by an auxiliary CNN. Central and right panels: quality of classification and discrimination by $\Psi_i$ and $\Psi_3$ with the same $D$ during the training process.
• Figure 3: Quality of sampling (left) and classification (right) by the trained $\Psi_3$ as a function of bond dimension.
• Figure 4: Normalized distributions $\rho(E)dE$ of energies $E(x)$ for images $x$ from the train (blue), sampled (red), and test (green) sets correspondingly. Solid lines are kernel density estimations.
• Figure 5: Typical images sampled with help of $\Psi_3$ with three different ranges of $E$, from left to right $E=60 \pm 5, 85 \pm 10, 125 \pm 10$.