Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum spectral method for gradient and Hessian estimation

Yuxin Zhang, Changpeng Shao

TL;DR

The paper introduces a quantum spectral method for gradient estimation of analytic functions that can take complex values, achieving an exponential speedup in dimension with a query complexity of $\widetilde{O}(1/\varepsilon)$, and extends this approach to Hessian estimation with polynomial speedups, including strong gains for sparse Hessians. The core idea is to express directional derivatives via a Cauchy-differentiation-based spectral formula $F(\mathbf{x})=\frac{1}{N\delta}\sum_{k} \omega^{-k} f(\delta \omega^{k}\mathbf{x})$, construct a phase oracle from $F$, and recover the gradient; Hessians are handled by reducing to gradient estimation, with analogous resource bounds. The results include two Hessian-estimation schemes under the spectral framework with $\widetilde{O}(d/\varepsilon)$ and $\widetilde{O}(d^{1.5}/\varepsilon)$ costs in the general case, and substantially improved $\widetilde{O}(s/\varepsilon)$ and $\widetilde{O}(sd/\varepsilon)$ costs for $s$-sparse Hessians, along with lower bounds of $\widetilde{\Omega}(d)$ for the general problem. The paper also analyzes a finite-difference approach for Hessians, highlighting the spectral method’s superior error behavior and its practical implications for Newton-type optimization in quantum settings. Overall, the work advances quantum subroutines for second-order optimization and clarifies the trade-offs between spectral and finite-difference strategies, especially in the presence of sparsity.

Abstract

Gradient descent is one of the most basic algorithms for solving continuous optimization problems. In [Jordan, PRL, 95(5):050501, 2005], Jordan proposed the first quantum algorithm for estimating gradients of functions close to linear, with exponential speedup in the black-box model. This quantum algorithm was greatly enhanced and developed by [Gilyén, Arunachalam, and Wiebe, SODA, pp. 1425-1444, 2019], providing a quantum algorithm with optimal query complexity $\widetildeΘ(\sqrt{d}/\varepsilon)$ for a class of smooth functions of $d$ variables, where $\varepsilon$ is the accuracy. This is quadratically faster than classical algorithms for the same problem. In this work, we continue this research by proposing a new quantum algorithm for another class of functions, namely, analytic functions $f(\boldsymbol{x})$ which are well-defined over the complex field. Given phase oracles to query the real and imaginary parts of $f(\boldsymbol{x})$ respectively, we propose a quantum algorithm that returns an $\varepsilon$-approximation of its gradient with query complexity $\widetilde{O}(1/\varepsilon)$. This achieves exponential speedup over classical algorithms in terms of the dimension $d$. As an extension, we also propose two quantum algorithms for Hessian estimation, aiming to improve quantum analogs of Newton's method. The two algorithms have query complexity $\widetilde{O}(d/\varepsilon)$ and $\widetilde{O}(d^{1.5}/\varepsilon)$, respectively, under different assumptions. Moreover, if the Hessian is promised to be $s$-sparse, we then have two new quantum algorithms with query complexity $\widetilde{O}(s/\varepsilon)$ and $\widetilde{O}(sd/\varepsilon)$, respectively. The former achieves exponential speedup over classical algorithms. We also prove a lower bound of $\widetildeΩ(d)$ for Hessian estimation in the general case.

Quantum spectral method for gradient and Hessian estimation

TL;DR

The paper introduces a quantum spectral method for gradient estimation of analytic functions that can take complex values, achieving an exponential speedup in dimension with a query complexity of , and extends this approach to Hessian estimation with polynomial speedups, including strong gains for sparse Hessians. The core idea is to express directional derivatives via a Cauchy-differentiation-based spectral formula , construct a phase oracle from , and recover the gradient; Hessians are handled by reducing to gradient estimation, with analogous resource bounds. The results include two Hessian-estimation schemes under the spectral framework with and costs in the general case, and substantially improved and costs for -sparse Hessians, along with lower bounds of for the general problem. The paper also analyzes a finite-difference approach for Hessians, highlighting the spectral method’s superior error behavior and its practical implications for Newton-type optimization in quantum settings. Overall, the work advances quantum subroutines for second-order optimization and clarifies the trade-offs between spectral and finite-difference strategies, especially in the presence of sparsity.

Abstract

Gradient descent is one of the most basic algorithms for solving continuous optimization problems. In [Jordan, PRL, 95(5):050501, 2005], Jordan proposed the first quantum algorithm for estimating gradients of functions close to linear, with exponential speedup in the black-box model. This quantum algorithm was greatly enhanced and developed by [Gilyén, Arunachalam, and Wiebe, SODA, pp. 1425-1444, 2019], providing a quantum algorithm with optimal query complexity for a class of smooth functions of variables, where is the accuracy. This is quadratically faster than classical algorithms for the same problem. In this work, we continue this research by proposing a new quantum algorithm for another class of functions, namely, analytic functions which are well-defined over the complex field. Given phase oracles to query the real and imaginary parts of respectively, we propose a quantum algorithm that returns an -approximation of its gradient with query complexity . This achieves exponential speedup over classical algorithms in terms of the dimension . As an extension, we also propose two quantum algorithms for Hessian estimation, aiming to improve quantum analogs of Newton's method. The two algorithms have query complexity and , respectively, under different assumptions. Moreover, if the Hessian is promised to be -sparse, we then have two new quantum algorithms with query complexity and , respectively. The former achieves exponential speedup over classical algorithms. We also prove a lower bound of for Hessian estimation in the general case.
Paper Structure (27 sections, 53 theorems, 148 equations, 1 figure)

This paper contains 27 sections, 53 theorems, 148 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Main results
  3. Summary of the key ideas
  4. Comparison with Gilyén-Arunachalam-Wiebe's algorithm
  5. Similarity
  6. Difference 1: Approximation methods
  7. Difference 2: Oracle settings
  8. Relevant previous results
  9. Outline of the paper
  10. Preliminaries
  11. Description of oracles
  12. Multivariable calculus
  13. Holomorphic functions in several complex variables
  14. Analytic functions
  15. Spectral method
  16. ...and 12 more sections

Key Result

Proposition 1.2

Assume that $f: \mathbb{R}^d \rightarrow \mathbb{R}$ is analytic and satisfies $\left|\partial^\alpha f(\boldsymbol{x})\right| \leq c^k k^{k/2}$ for every ${\boldsymbol{x}}\in \mathbb{R}^d$, $k \in \mathbb{N}$ and $\alpha \in \mathbb N_0^d$ such that $|\alpha|=k$. Given the phase oracle $O_f$, there

Figures (1)

  • Figure 1: Comparison between GAW's formula and ours. The figure illustrates the points that are used to approximate the derivative for functions of dimension $d=1$. The blue points on the $x$ axis are used in the higher-order finite difference method and the red points on the circle are used in the spectral method.

Theorems & Definitions (102)

  • Proposition 1.2: Finite difference method
  • Theorem 1.3: Informal form of Theorem \ref{['thm: gradient estimation using spectral method']}, spectral method
  • Remark 1.4
  • Theorem 1.5: Informal form of Theorem \ref{['thm: hessian using spectral']}, spectral method
  • Theorem 1.6: Informal form of Theorem \ref{['new thm: Hessian estimation using finite difference formula']}, finite difference method
  • Theorem 1.7: Informal forms of Propositions \ref{['prop: lower bound']} and \ref{['prop: lower bound of real-valued Hessian estimation']}
  • Theorem 1.8: Informal form of Theorem \ref{['thm: Sparse Hessian estimation using spectral method']}, spectral method
  • Theorem 1.9: Informal form of Theorem \ref{['thm: Sparse Hessian estimation using finite difference formula']}, finite difference method
  • Lemma 1.10
  • Remark 1.11
  • ...and 92 more