Solving the Hubbard model with Neural Quantum States

TL;DR

The paper tackles the challenge of solving the doped 2D Hubbard model, a prototypical strongly correlated fermionic system, by developing a transformer-based Neural Quantum State (NQS) within variational Monte Carlo and introducing the Moment-Adaptive ReConfiguration Heuristic (MARCH) optimizer along with pretraining and a pinning-field technique. This framework achieves state-of-the-art variational energies up to 16×16 lattices, exhibits stripe-ordered ground states consistent with cuprate experiments, and reveals that different attention heads encode correlations at multiple scales. The work demonstrates that NQS can surpass traditional methods like DMRG and PEPS in large 2D settings under periodic boundary conditions, and establishes a scalable, physics-aware approach for studying challenging many-fermion systems. The combination of architecture, optimization, and stabilization strategies positions NQS as a powerful tool for exploring strongly correlated quantum matter and related phenomena.

Abstract

The rapid development of neural quantum states (NQS) has established it as a promising framework for studying quantum many-body systems. In this work, by leveraging the cutting-edge transformer-based architectures and developing highly efficient optimization algorithms, we achieve the state-of-the-art results for the doped two-dimensional (2D) Hubbard model, arguably the minimum model for high-Tc superconductivity. Interestingly, we find different attention heads in the NQS ansatz can directly encode correlations at different scales, making it capable of capturing long-range correlations and entanglements in strongly correlated systems. With these advances, we establish the half-filled stripe in the ground state of 2D Hubbard model with the next nearest neighboring hoppings, consistent with experimental observations in cuprates. Our work establishes NQS as a powerful tool for solving challenging many-fermions systems.

Figures (39)

• Figure 1: Application, architecture, optimization, and performance of Neural Quantum States (NQS) on the Hubbard model. (A) Guided by the variational principle, NQS are powerful and versatile, applicable to a wide range of quantum systems, including ab initio quantum chemistry, spin models, and correlated fermionic models. (B) Various neural network architectures can be used to construct NQS, with increasing expressivity from Restricted Boltzmann Machines (RBM) nomura2017restricted, Multi-Layer Perceptrons (MLP) inui2021determinantluo2019backflowrobledo2022fermioniczhou2024solving to Transformers. These networks are optimized using methods inspired by quantum information, including Min-SR chen2024empoweringrende2024simple, SPRING goldshlager2024kaczmarz, and the advanced MARCH method proposed in this work. (C) A performance comparison of different NQS methods on the 2D Hubbard model, plotting simulation accuracy against system size. We benchmark the accuracy on systems where exact solutions are available (see Supplementary Section \ref{['sec:acc_estimate']} for details). This work achieves higher accuracy on larger system sizes than previously reported in other notable NQS studies, demonstrating a significant advancement in the field.
• Figure 2: NQS architecture, optimization scheme, and benchmark performance on the Hubbard model. (A) An input lattice configuration is processed through a transformer-based neural network to generate backflow orbitals. These orbitals are then used to construct the NQS wavefunction $\psi _\theta$. The variational parameters $\theta$ are optimized to find the ground state. (B) Optimization trajectories for MARCH and SPRING. MARCH follows a more direct path by dynamically adapting its step size for each parameter. The inset shows how SPRING's reliance on momentum might lead to overshooting, whereas MARCH's adaptive-momentum step prevents oscillations and accelerating convergence to the minimum. (C) Comparison of ground-state energies with PEPS (bond dimension $D \geq 20)$liu2025 for the pure Hubbard model on $16\times L_y$ system with OBC. Our NQS results consistently achieve lower variational energies than PEPS across different lattice geometries. The inset shows we also outperform reference DMRG energies liu2025 (with $32000$ SU(2) multiplets) when $L_y = 8$.
• Figure 3: Visualization of the attention mechanism. Shown is how a lattice site attends to another target site in the 2D pure Hubbard model for lattice sizes of $8\times 8$, $10 \times 10$, and $12 \times 12$ at half filling. The four attention heads, which are equivalent and have been rearranged by pattern for clarity, each correspond to a specific physical aspect: short-range correlations, nearest-neighbor hopping, antiferromagnetic correlations, and complex and delocalized correlations in the ground state.
• Figure 4: Hole and spin density distributions in the ground state for different lattice sizes with $U = 8$ and hole doping $\delta = 1/8$ under PBC in the Hubbard model. The magnitudes of the spin density are represented by the sizes of the arrows while the direction is denoted by the direction of the arrows. Hole density is depicted using a color scale. Stripes can be clearly seen in all three systems. (A) Results for $16 \times 16$ pure Hubbard model where the stripe state with $\lambda=8$ is observed. (B) Results for $16 \times 12$ system with $t'=-0.2$ where the stripe state with $\lambda=4$ is observed. (C) Results for $32 \times 8$ system with $t'=-0.2$ where the stripe state with $\lambda=4$ is observed. Notice that the stripes in the ground state for $t^\prime = -0.2$ in (B) and (C) are horizontal, even though both vertical and horizontal stripes are allowed in the systems. More results can be found in the supplementary materials.
• Figure 5: Energy difference between vertical stripe and horizontal stripe. Kinetic energy, potential energy, and total energy results are all shown. We systematically study systems $L_x \times 8$ with $L_x$ ranging from 8 to 32, with $t^\prime=-0.2$ and $\delta =1/8$. We can find that the horizontal stripe has lower energy for all the systems (notice that for $8 \times 8$ system, vertical and horizontal stripes are the same). Moreover, the results show that the gain of energy in the horizontal stripe is mainly from the kinetic energy.