Forward-mode automatic differentiation for the tensor renormalization group and its relation to the impurity method

TL;DR

This work introduces forward-mode automatic differentiation integrated with tensor renormalization group methods to compute derivatives of the partition function and observables directly along the renormalization flow. By propagating derivative information with the forward TRG steps, the method achieves derivatives up to order $k$ at a contraction-cost factor of $(k+1)(k+2)/2$ and memory growth of $k$, avoiding the heavy memory burden of reverse-mode AD. The framework reproduces the impurity method in the SVD-derivative-vanishing limit, while delivering significantly higher accuracy for internal energy and specific heat in 2D (and reasonable results in 3D) and enabling finite-size scaling analyses to extract critical exponents from derivatives of renormalized tensors. The approach applies to HOTRG and BWTRG and provides a practical path for derivative-based investigations in tensor-network calculations, with potential extensions to higher dimensions, symmetry-aware implementations, and dynamical observables.

Abstract

We propose a forward-mode automatic differentiation (AD) framework for tensor renormalization group (TRG) methods. In this approach, evaluating the derivatives of the partition function up to order $k$ increases the matrix-multiplication cost by a factor of $(k+1)(k+2)/2$ compared to computing the free energy alone, while the memory footprint is only $k$ times that of the original calculation. In the limit where the derivatives of the SVD are neglected, we establish a theoretical correspondence between our forward-mode AD and conventional impurity methods. Numerically, we find that the proposed AD algorithm can calculate internal energy and specific heat significantly higher accuracy than the impurity method at comparable computational cost. We also provide a practical procedure to extract critical exponents from derivatives of the renormalized tensor in TRG calculations in both two and three dimensions.

Figures (8)

• Figure 1: Relative errors of (a) the internal energy and (b) the specific heat obtained from second-order forward-mode AD within HOTRG at $D=80$. The horizontal axis shows the temperature. Results are shown for several regularization parameters $\eta$ and are compared with the impurity method.
• Figure 2: Comparison of the elapsed time of the coarse-graining part of the forward-mode AD and the impurity-tensor method as a function of $D$. $\tau_{\mathrm{AD/impurity}}$ denotes the computational time of the AD and impurity method, respectively. $\tau_{\mathrm{HOTRG}}$ denotes the computational time of the HOTRG method alone. (a) Elapsed time of the bottleneck contraction part for obtaining up to the first-order derivative, normalized by the original HOTRG contraction time without derivatives. (b) Total elapsed time of HOTRG computations up to the second derivatives, normalized by the execution time of the original HOTRG. The dotted line indicates theoretical scaling factor $\frac{(k+1)(k+2)}{2}$.
• Figure 3: Elapsed time comparison between the forward-mode AD and the impurity-tensor method of the HOTRG method as a function of $D$. $\tau_{\mathrm{AD/impurity}}$ denotes the computational time of the AD and impurity method, respectively. $\tau_{\mathrm{HOTRG}}$ denotes the computational time of the usual HOTRG method alone. (a) Elapsed time of the obtaining up to first derivatives on a $V=2^{40}$ lattice, normalized by the original HOTRG contraction time without derivatives. (b) Total elapsed time of HOTRG computations up to the second derivatives on a $V=2^{40}$ lattice, normalized by the execution time of the original HOTRG. The dotted line indicates theoretical scaling factor $\frac{(k+1)(k+2)}{2}$.
• Figure 4: Relative error of the internal energy with respect to the exact result as a function of temperature. Circles show the forward-mode AD data for various bond dimensions. The impurity limit corresponds to setting the SVD derivatives to zero and using the bond-based initial tensor described in Sec. \ref{['subsec:BWTRG']}; these data are shown by square symbols. The multi-impurity data from Ref. moritaMultiimpurityMethodBondweighted2024 are also shown by black symbols. The dotted line indicates the exact critical temperature.
• Figure 5: Two-dimensional Ising model computed using the forward-mode AD BWTRG at $D=130$. (a) $L$ dependence of $\left|\pdv{X}{T}\right|_{T=T_c}$. (b) $L$ dependence of $1/\nu_{\mathrm{eff},i}$.