Approaching the Thermodynamic Limit with Neural-Network Quantum States

TL;DR

This work introduces Spatial Attention, a minimal, physically grounded bias in Transformer-based Neural-Network Quantum States that encodes a learnable length scale to bias correlations by distance. The approach stabilizes optimization on large two-dimensional lattices and enables controlled finite-size scaling to access thermodynamic-limit physics in frustrated quantum magnets, demonstrated on the triangular-lattice Heisenberg model where energies surpass prior variational methods and a record low magnetization $M_0=0.148(1)$ is obtained. The study also reveals that the ground-state sign structure is intrinsically non-local, with sign overlaps decaying exponentially with system size, suggesting a genuine sign problem that cannot be removed by local basis changes. Beyond the triangular lattice, the method achieves state-of-the-art results for the $J_1$-$J_2$ Heisenberg model on a $20\times20$ square lattice, illustrating the approach's generality and potential applicability to broader quantum many-body problems, including fermionic systems via backflow. Overall, Spatial Attention provides a scalable variational framework that brings large-system, thermodynamic-limit analyses of frustrated quantum magnets within practical reach.

Abstract

Accessing the thermodynamic-limit properties of strongly correlated quantum matter requires simulations on very large lattices, a regime that remains challenging for numerical methods, especially in frustrated two-dimensional systems. We introduce the Spatial Attention mechanism, a minimal and physically interpretable inductive bias for Neural-Network Quantum States, implemented as a single learned length scale within the Transformer architecture. This bias stabilizes large-scale optimization and enables access to thermodynamic-limit physics through highly accurate simulations on unprecedented system sizes within the Variational Monte Carlo framework. Applied to the spin-$\tfrac12$ triangular-lattice Heisenberg antiferromagnet, our approach achieves state-of-the-art results on clusters of up to $42\times42$ sites. The ability to simulate such large systems allows controlled finite-size scaling of energies and order parameters, enabling the extraction of experimentally relevant quantities such as spin-wave velocities and uniform susceptibilities. In turn, we find extrapolated thermodynamic limit energies systematically better than those obtained with tensor-network approaches such as iPEPS. The resulting magnetization is strongly renormalized, $M_0=0.148(1)$ (about $30\%$ of the classical value), revealing that less accurate variational states systematically overestimate magnetic order. Analysis of the optimized wave function further suggests an intrinsically non-local sign structure, indicating that the sign problem cannot be removed by local basis transformations. We finally demonstrate the generality of the method by obtaining state-of-the-art energies for a $J_1$-$J_2$ Heisenberg model on a $20\times20$ square lattice, outperforming Residual Convolutional Neural Networks.

Figures (8)

• Figure 1: Schematic of the Spatial Attention mechanism. For each input spin, an abstract representation is formed by aggregating contributions from all other spins, weighted by their spatial separation: nearby spins contribute more strongly, while contributions from distant sites are progressively suppressed. This operation is performed independently at every lattice site, yielding a new representation for each spin. The procedure is iterated across successive Transformer layers.
• Figure 2: Energy scaling and size-consistent accuracy.a. Finite-size extrapolations of the ground-state variational energy per site as a function of $1/L^{3}$, using data from lattice sizes $L = 24, 30,$ and $42$. Results obtained with the fully symmetrized ViT wave function are shown as violet triangles, while red rhombi denote energies obtained from zero-variance extrapolations (see Sec. \ref{['sec:en_var']} for details). These results are compared with the most recent and accurate thermodynamic-limit estimates from iPEPS calculations, including iPEPS TL naumann2025 and iPEPS $U(1)$Hasik2024. Extrapolated energies are not shown for methods whose best variational energy exceeds $-0.551$. b. Top panel: relative error of the variational energies with respect to the zero-variance extrapolated values as a function of system size $L$, for different symmetry projections: no symmetry projection (No symm, blue rhombi), translation symmetry projection (Symm $T$, orange circles), and full translation and rotation symmetry projection (Symm $T+R$, violet triangles). Bottom panel: corresponding V-score values.
• Figure 3: Size-scaling of the magnetization.a. Finite-size extrapolation of the magnetization order parameter, as defined in in the main text, to the thermodynamic limit (violet triangles). All the available results in literature are showed for comparison (gray triangles) and reported in \ref{['table:magnetization']} in \ref{['sec:magn_lit']}. Inset: Zoom of the same quantity for $1/L \rightarrow 0$ in which different extrapolations are shown, quadratic (solid line) and linear (dashed line). b. Magnetization order parameter as a function of optimization steps for the $L=30$ cluster, comparing the fully symmetrized ViT (violet horizontal line) with the non-symmetrized ViT (blue rhombi). Reference values of the ground-state variational energies at selected optimization steps and the final energy for the fully symmetrized state are also shown.
• Figure 4: Ground-state sign structure.a. Mean sign overlap [see definition in \ref{['eq:mean_sign']}] between the ViT state and the Huse–Elser prescription huseCapriotti1999 as a function of its variational parameter $\beta$. Exact results are shown for $L = 6$, for which the sign overlap between the ViT wave function and the exact ground state is ${\langle s \rangle = 0.99999707}$. b. Mean sign overlap as a function of the number of sites $L^2$ between the Transformer wave function and the classical sign structure (green circles) or the Huse–Elser wave function huse at the optimal $\beta \approx 0.27$ (blue circles).
• Figure 5: Energy $J_1$-$J_2$ Heisenberg model. Variational energies for $J_1$-$J_2$ Heisenberg model at $J_2/J_1=0.5$ on a $20\times 20$ cluster with periodic boundary conditions plotted as a function of the variance per site. The three blue empty points correspond to ViT wave functions with: no symmetry, translational symmetry, and full point-group symmetry restoration. A linear extrapolation of these points to zero variance yields an energy of $-0.49684(1)$ (blue full circle). The green circle show the variational energy obtained with the ResNet$2$ Ansatz with one Lanczos step, together with the zero-variance extrapolated value (see dashed green line) reported in Ref. chen2024empowering.