Quantum Algorithms and Lower Bounds for Finite-Sum Optimization
Yexin Zhang, Chenyi Zhang, Cong Fang, Liwei Wang, Tongyang Li
TL;DR
This work initiates quantum algorithms for finite-sum optimization by introducing a quantum finite-sum oracle that can query gradients in superposition and studying four convex configurations of $F(x)=\frac{1}{n}\sum_i f_i(x)+\psi(x)$. The authors develop quantum variants of Katyusha and SPIDER that replace classical variance-reduction steps with unbiased quantum mean estimation via quantum variance reduction, achieving improved query complexities such as $\tilde{O}\big(n+\sqrt{d}+\sqrt{\ell/\mu}(n^{1/3}d^{1/3}+n^{-2/3}d^{5/6})\big)$ in the strongly convex case and a nonconvex analogue $\tilde{O}\big(n+(d^{1/3}n^{1/3}+\sqrt{d})/\epsilon^{2}\big)$ for ε-critical points. The paper also proves quantum lower bounds for each case using the quantum adversary method by reducing to multi-chain and matrix-detection problems, showing polynomial limits on speedups. A key technical ingredient is an unbiased quantum mean estimator built via MLMC, enabling integration with Katyusha/SPIDER and ensuring convergence guarantees. The results extend to non-smooth or non-strongly convex variants and provide a quantum framework for finite-sum optimization, with open questions on tighter bounds and practical ML applications.
Abstract
Finite-sum optimization has wide applications in machine learning, covering important problems such as support vector machines, regression, etc. In this paper, we initiate the study of solving finite-sum optimization problems by quantum computing. Specifically, let $f_1,\ldots,f_n\colon\mathbb{R}^d\to\mathbb{R}$ be $\ell$-smooth convex functions and $ψ\colon\mathbb{R}^d\to\mathbb{R}$ be a $μ$-strongly convex proximal function. The goal is to find an $ε$-optimal point for $F(\mathbf{x})=\frac{1}{n}\sum_{i=1}^n f_i(\mathbf{x})+ψ(\mathbf{x})$. We give a quantum algorithm with complexity $\tilde{O}\big(n+\sqrt{d}+\sqrt{\ell/μ}\big(n^{1/3}d^{1/3}+n^{-2/3}d^{5/6}\big)\big)$, improving the classical tight bound $\tildeΘ\big(n+\sqrt{n\ell/μ}\big)$. We also prove a quantum lower bound $\tildeΩ(n+n^{3/4}(\ell/μ)^{1/4})$ when $d$ is large enough. Both our quantum upper and lower bounds can extend to the cases where $ψ$ is not necessarily strongly convex, or each $f_i$ is Lipschitz but not necessarily smooth. In addition, when $F$ is nonconvex, our quantum algorithm can find an $ε$-critial point using $\tilde{O}(n+\ell(d^{1/3}n^{1/3}+\sqrt{d})/ε^2)$ queries.