Essentially No Energy Barrier Between Independent Fermionic Neural Quantum State Minima

TL;DR

The paper investigates whether neural quantum states (NQS) exhibit rugged loss landscapes by demonstrating mode connectivity between independently trained Psiformers using GeoNEB, a QGT-informed Nudged Elastic Band algorithm. It shows two 1.6M-parameter Psiformers for a six-electron quantum dot connected by a nonlinear path with a tiny barrier of $7.2\times10^{-4}$ Ha while maintaining angular momentum quantization, indicating a much more benign landscape than feared. The approach combines exact stochastic reconfiguration (natural gradient) with an AutoNEB-inspired path optimization, including dynamic pivot insertion and force nudging to minimize the maximum energy along the path. These results imply that NQS optimization can be robust and transferable, with significant implications for scaling NQS methods to larger quantum systems.

Abstract

Neural quantum states (NQS) have proven highly effective in representing quantum many-body wavefunctions, but their loss landscape remains poorly understood and debated. Here, we demonstrate that the NQS loss landscape is more benign and similar to conventional deep learning than previously thought, exhibiting mode connectivity: independently trained NQS are connected by paths in parameter space with essentially no energy barrier. To construct these paths, we develop GeoNEB, a path optimizer integrating efficient stochastic reconfiguration with the nudged elastic band method for constructing minimum energy paths. For the strongly interacting six-electron quantum dot modeled by a $1.6$M-parameter Psiformer, we find two independent minima with expected energy barrier $\sim10^{-5}$ times smaller than the system's overall energy scale and $\sim10^{-3}$ times smaller than the linear path's barrier. The path respects physical symmetry in addition to achieving low energy, with the angular momentum remaining well quantized throughout. Our work is the first to construct optimized paths between independently trained NQS, and it suggests that the NQS loss landscape may not be as pathological as once feared.

Figures (7)

• Figure 1: Schematic of the Psiformer architecture. Electron coordinates are embedded and processed through transformer layers. A determinant enforces antisymmetry, while Jastrow and envelope factors handle cusps and boundary conditions.
• Figure 2: Pseudocode for GeoNEB. The curve consists of coupled NQS instances: each maintains its own Markov chains and local quantum geometric tensor but interacts with its neighbors to calculate the spring force and tangent.
• Figure 3: Comparison of Energy Barriers. GeoNEB finds a path with maximum barrier $7.2\times10^{-4}\text{ Ha}$, linear interpolation's barrier is $1.86\text{ Ha}$, and the ground state energy is $40.817\text{ Ha}$. The endpoints have Euclidean parameter norms $94.8$ and $95.1$, while their separation has norm $115.9$, confirming that the endpoints are far apart. Every fourth point is a trainable pivot; the others are inference-only.
• Figure 4: Comparison of Energy Variance. Plot uses log scale for visibility. GeoNEB's peak is $1.1\times10^{-3}\text{ Ha}^2$ while linear interpolation's peak is $3.6\text{ Ha}^2$. Every fourth point is a trainable pivot; the others are inference-only.
• Figure 5: Comparison of Angular Momentum Variance. Plot uses log scale for visibility. GeoNEB's peak is $6.5\times10^{-4}\text{ a.u.}^2$ while linear interpolation's peak is $0.61\text{ a.u.}^2$. Every fourth point is a trainable pivot; the others are inference-only. The GeoNEB barrier is nontrivial, but in an absolute sense $6.5\times10^{-4}\text{ a.u.}^2$ is still good performance and the difference reflects the endpoints' excellence, not the path's shortcoming.