Implicitly Restarted Lanczos Enables Chemically-Accurate Shallow Neural Quantum States

TL;DR

This work introduces implicitly restarted Lanczos (IRL) as a robust, second-order optimizer for neural quantum states (NQS). By reformulating the variational parameter update from the ill-conditioned SR/LM linear system into a Hermitian eigenproblem and solving it with IRL, the method automatically determines the descent direction and step size, removing the need for learning-rate tuning and mitigating numerical instability. The approach enables shallow NQS to achieve chemical accuracy ($\varepsilon_{\text{abs}} \sim 10^{-10}$ to $10^{-12}$ kcal/mol) in 3–5 optimization steps, with dramatic speedups (e.g., ~17,900x over Adam for F$_2$) and parameter efficiency. Crucially, IRL remains stable and accurate in strongly correlated regimes (e.g., N$_2$ bond dissociation), where traditional single-reference methods fail, suggesting broad applicability to high-precision quantum chemistry and quantum physics leveraging variational neural networks.

Abstract

The variational optimization of high-dimensional neural network models, such as those used in neural quantum states (NQS), presents a significant challenge in machine intelligence. Conventional first-order stochastic methods (e.g., Adam) are plagued by slow convergence, sensitivity to hyperparameters, and numerical instability, preventing NQS from reaching the high accuracy required for fundamental science. We address this fundamental optimization bottleneck by introducing the implicitly restarted Lanczos (IRL) method as the core engine for NQS training. Our key innovation is an inherently stable second-order optimization framework that recasts the ill-conditioned parameter update problem into a small, well-posed Hermitian eigenvalue problem. By solving this problem efficiently and robustly with IRL, our approach automatically determines the optimal descent direction and step size, circumventing the need for demanding hyperparameter tuning and eliminating the numerical instabilities common in standard iterative solvers. We demonstrate that IRL enables shallow NQS architectures (with orders of magnitude fewer parameters) to consistently achieve extreme precision (1e-12 kcal/mol) in just 3 to 5 optimization steps. For the F2 molecule, this translates to an approximate 17,900-fold speed-up in total runtime compared to Adam. This work establishes IRL as a superior, robust, and efficient second-order optimization strategy for variational quantum models, paving the way for the practical, high-fidelity application of neural networks in quantum physics and chemistry.

Figures (3)

• Figure 1: Schematic of the IRL-Optimized NQS Optimization Framework. a, The traditional first-order stochastic optimization landscape (e.g., Adam) is noisy and requires careful tuning of the learning rate ($\eta$). b, Our approach reformulates the second-order parameter update as an eigenvalue problem in the neural network's parameter space. c, The IRL method stably solves this problem, finding the optimal descent direction (red arrow) and step size automatically, enabling robust navigation of the optimization landscape.
• Figure 2: Optimization landscape and trajectories for different NQS training methods, with a base neural network size of 5,622 parameters.(a) Three-dimensional representation of the variational energy error surface $\varepsilon_{\text{abs}}=|E_{\text{NQS}}-E_{\text{FCI}}|$ as a function of two shifted hyperparameters $\theta_1$ and $\theta_2$, with the global minimum corresponding to FCI accuracy (the origin (0,0) is shifted to align with the global minimum for visualization clarity). (b) Two-dimensional projection with optimization paths for Adam, standard Lanczos (SL), and implicitly restarted Lanczos (IRL) methods. (c) Convergence behavior showing IRL reaching the global minimum in only 5 steps (5.1 milliseconds), compared to 10,000 Adam steps (91.2 seconds) and 50 SL steps (0.272 seconds) that fail to converge. Data are from $\text{F}_2$ molecule calculations. (d) Performance scaling with model size for the $\text{F}_2$ molecule. As the network dimension (e.g., number of hidden units or layers) increases, Adam's optimization behavior exhibits a clear scaling trend. In contrast, both SL and IRL demonstrate robust stability, showing minimal dependence on the model's scale. All calculations were performed with a fixed random seed of 111.
• Figure 3: Bond dissociation curve of the N$_2$ molecule computed using various electronic structure methods, including Hartree–Fock, CCSD(T), FCI, and the IRL-NQS approach introduced in this work. The total energy (in Hartree) is plotted as a function of the N–N bond length (in Å). The IRL-NQS results demonstrate close agreement with high-accuracy FCI calculations across the dissociation range.