Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Isotropic Noise in Stochastic and Quantum Convex Optimization

Annie Marsden, Liam O'Carroll, Aaron Sidford, Chenyi Zhang

TL;DR

This work introduces isotropic noise as a structured shape constraint for stochastic gradient oracles in convex SCO and develops a dimension-aware algorithm achieving $\tilde{O}(R^2\sigma_I^2/\epsilon^2 + d)$ queries under ISGO, and $\tilde{O}(R^2\sigma_E^2/\epsilon^2 + d)$ under sub-exponential ESGO noise, with a separate $\tilde{O}(dR^2\sigma_V^2/\epsilon^2 + d)$ bound under VSGO. It proves matching lower bounds up to polylog factors via reductions from isotropic mean-estimation problems, establishing near-optimality in the ISGO/ESGO regime and near-linear scaling in the dimension. A key technical contribution is the quantum isotropifier, which converts a quantum variance-bounded gradient oracle into an unbiased, isotropic gradient estimator, enabling improved quantum SCO rates of approximately $\tilde{O}(dR\sigma_V/\epsilon)$. Collectively, the results advance a nuanced understanding of how noise shape and dimension interact in SCO and its quantum analogue, with implications for efficient algorithms under light-tailed noise and for quantum speedups in stochastic optimization.

Abstract

We consider the problem of minimizing a $d$-dimensional Lipschitz convex function using a stochastic gradient oracle. We introduce and motivate a setting where the noise of the stochastic gradient is isotropic in that it is bounded in every direction with high probability. We then develop an algorithm for this setting which improves upon prior results by a factor of $d$ in certain regimes, and as a corollary, achieves a new state-of-the-art complexity for sub-exponential noise. We give matching lower bounds (up to polylogarithmic factors) for both results. Additionally, we develop an efficient quantum isotropifier, a quantum algorithm which converts a variance-bounded quantum sampling oracle into one that outputs an unbiased estimate with isotropic error. Combining our results, we obtain improved dimension-dependent rates for quantum stochastic convex optimization.

Isotropic Noise in Stochastic and Quantum Convex Optimization

TL;DR

This work introduces isotropic noise as a structured shape constraint for stochastic gradient oracles in convex SCO and develops a dimension-aware algorithm achieving queries under ISGO, and under sub-exponential ESGO noise, with a separate bound under VSGO. It proves matching lower bounds up to polylog factors via reductions from isotropic mean-estimation problems, establishing near-optimality in the ISGO/ESGO regime and near-linear scaling in the dimension. A key technical contribution is the quantum isotropifier, which converts a quantum variance-bounded gradient oracle into an unbiased, isotropic gradient estimator, enabling improved quantum SCO rates of approximately . Collectively, the results advance a nuanced understanding of how noise shape and dimension interact in SCO and its quantum analogue, with implications for efficient algorithms under light-tailed noise and for quantum speedups in stochastic optimization.

Abstract

We consider the problem of minimizing a -dimensional Lipschitz convex function using a stochastic gradient oracle. We introduce and motivate a setting where the noise of the stochastic gradient is isotropic in that it is bounded in every direction with high probability. We then develop an algorithm for this setting which improves upon prior results by a factor of in certain regimes, and as a corollary, achieves a new state-of-the-art complexity for sub-exponential noise. We give matching lower bounds (up to polylogarithmic factors) for both results. Additionally, we develop an efficient quantum isotropifier, a quantum algorithm which converts a variance-bounded quantum sampling oracle into one that outputs an unbiased estimate with isotropic error. Combining our results, we obtain improved dimension-dependent rates for quantum stochastic convex optimization.
Paper Structure (33 sections, 42 theorems, 156 equations)

This paper contains 33 sections, 42 theorems, 156 equations.

Table of Contents

  1. Introduction
  2. Results
  3. Isotropic and sub-exponential SGOs.
  4. The complexity of SCO with ISGOs, ESGOs, and VSGOs.
  5. Quantum SCO under variance-bounded noise.
  6. Additional related work
  7. Stochastic convex optimization.
  8. Quantum mean estimation.
  9. Quantum algorithms and lower bounds for optimization problems.
  10. Notation
  11. Paper organization
  12. Technical overview
  13. Overview of our upper bounds for SCO
  14. Overview of our lower bounds for SCO
  15. Quantum isotropifier and quantum SCO under variance-bounded noise
  16. ...and 18 more sections

Key Result

Theorem 1

Problem prob:SCO can be solved with probability at least $2/3$ in $\tilde{O} \inparen*{R^2 \sigma_I^2 / \epsilon^{2} + d}$ queries to a $(\sigma_I,\delta)$-ISGO for any $\delta \le \frac{1}{M d (R^2 \sigma_I^2 / \epsilon^{2} + d)}$, where $M = \tilde{O}(1)$.

Theorems & Definitions (83)

  • Definition 1: SGO
  • Definition 2: BSGO
  • Definition 3: VSGO
  • Definition 4: ISGO
  • Definition 5: ESGO
  • Theorem 1
  • Corollary 1
  • Corollary 1
  • Corollary 1
  • Theorem 2
  • ...and 73 more