Transformer Neural-Network Quantum States for lattice models of spins and fermions: Application to the Ancilla Layer Model

TL;DR

These findings establish Transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight their potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.

Abstract

We introduce a variational wave function based on Neural-Network Quantum States (NQS) to study lattice systems whose local Hilbert space contains both spin and fermionic degrees of freedom. Our approach is based on the use of the Transformer architecture, which can naturally handle composite local Hilbert spaces through a tokenization procedure closely inspired by techniques from natural language processing. The neural network predicts a set of fermionic orbitals that depend on the spin configuration in a backflow-inspired manner. We apply the method to the one-dimensional Ancilla Layer Model, consisting of a chain of mobile spin-$1/2$ fermions coupled to a two-leg spin-$1/2$ ladder. For open boundary conditions, we achieve excellent quantitative agreement with Density Matrix Renormalization Group (DMRG) results across the full range of parameters considered. We find a phase in which the chain forms an effectively decoupled Luttinger liquid (LL), and a LL* phase with a distinct Fermi wavevector in which the mobile fermions are Kondo screened by one leg of the ladder, while the other leg forms the critical Bethe spin liquid. The LL* is the analog of the phase describing the pseudogap in two dimensions. We also find a Luther-Emery (LE) phase, where the LL* state becomes unstable toward the formation of a spin gap. The Transformer Ansatz maintains comparable accuracy for periodic boundary conditions, where tensor-network methods are computationally more demanding. Together, these findings establish Transformer-based NQS as an accurate and scalable variational framework for correlated lattice systems with composite local Hilbert spaces and highlight their potential for studying higher-dimensional models where boundary effects and heterogeneous local structures pose significant challenges.

Figures (9)

• Figure 1: Panel a: Schematic representation of the one-dimensional Ancilla Layer Model [see \ref{['eq:hamiltonian']}]. A chain of itinerant spinful fermions (blue arrows) of density $1-\delta$ is locally coupled via a Kondo exchange $J_K$ to the first leg of a spin-$\tfrac{1}{2}$ ladder (green arrows). The two ancillary spin layers are coupled along the chain direction by Heisenberg interactions $J_1$ and $J_2$, respectively, and across the rungs by an interlayer coupling $J_\perp$. Each lattice site thus hosts both fermionic and spin degrees of freedom, resulting in a composite local Hilbert space. Panel b: Each lattice site hosts two ancillary spin-$\tfrac{1}{2}$ variables and a spinful fermionic occupation, giving a composite local Hilbert space of dimension $\mathcal{V}=2^4$.
• Figure 2: Schematic representation of two phases of the one-dimensional ALM. Panel a: The mobile fermions form a Luttinger liquid with the conventional Fermi wavevector, $k_F$, of a decoupled chain, while the spin ladder is smoothly connected to the trivial, gapped rung-singlet phase. Panel b: The mobile fermions are Kondo screened, and the Fermi wavevector, $k_F^\ast$, is that of a Kondo lattice model; the bottom leg of the spin ladder forms a critical spin liquid with the same critical singularities as those of the decoupled spin-$1/2$ chain solved by Bethe.
• Figure 3: Phase diagram of the one-dimensional ALM as a function of the Kondo coupling $J_K$ and the interlayer exchange $J_\perp$. The gray region is the LL phase (C1S1) with central charge $c=2$, the red region is the LL$^*$ phase (C1S2) with $c=3$, while the blue region is the Luther Emery phase with $c=1$ (C1S0). The dashed white line denotes the boundary of the Kondo screened phase with peak at $2k_F^*$ in spin-spin correlation functions (refer to \ref{['sec:energy_corr']}). The central charge is obtained from iDMRG calculations of entanglement entropy ($\chi=500,1000$) with a unit cell of length $L=14$ with number of electrons $N_e=10$ for fixed values of $t=1.0$ and $J_1=J_2=0.5$.
• Figure 4: A physical configuration $\boldsymbol{s}=(s_1,\ldots,s_N)$, with $s_i=(n_{i\uparrow},n_{i\downarrow},S^z_{1i},S^z_{2i})$, is first encoded as a sequence of integer tokens $(t_1,\ldots,t_N)$, where $t_i\in\{0,\ldots,\mathcal{V}-1\}$ and $\mathcal{V}$ is the local Hilbert-space dimension. The tokens are embedded into feature vectors $(\boldsymbol{x}_1,\ldots,\boldsymbol{x}_N)$ and processed by a Transformer to produce context-dependent outputs $(\boldsymbol{y}_1,\ldots,\boldsymbol{y}_N)$. These outputs parameterize the backflow single-particle orbitals $\Phi(\boldsymbol{s})$, and the many-body amplitude $\Psi_\theta(\boldsymbol{s})$ is obtained via a Slater-determinant construction.
• Figure 5: Variational energy per site as a function of the energy variance per site on a chain of $N=42$ sites with OBC. The couplings are $t=1.0$, $J_1=J_2=J_\perp=0.5$, $J_K=4.0$ and the number of electrons is $N_e=30$ ($\delta \approx 0.2857$). Results are shown for Transformer wave functions with different number of layers $n_l=2,4,6$ (refer to \ref{['sec:architecture']} for details). The dashed line indicates the DMRG reference energy obtained with bond dimension $\chi=10^3$. Inset: Relative energy error $\Delta\varepsilon = \tfrac{|E_{\text{NQS}} - E_{\text{DMRG}}|}{|E_{\text{DMRG}}|}$ as a function of the Kondo coupling $J_K$, considering a Transformer with $n_l=4$ layers.