Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization

Shaocong Ma, Heng Huang

TL;DR

The paper solves a long-standing bias issue in zeroth-order optimization by constructing unbiased gradient estimators that rely solely on function evaluations. It achieves this through a telescoping-series representation of directional derivatives and a carefully designed sampling scheme over perturbation indices, yielding four concrete estimators that trade off function-query cost and variance. The authors prove unbiasedness, characterize variance bounds, and identify optimal choices for perturbation sequences and sampling distributions, showing that SGD with these estimators attains the optimal $\mathcal{O}(d/\epsilon^4)$ complexity for smooth non-convex objectives. Empirically, the approach yields lower gradient-estimation error and faster convergence in synthetic tasks and language-model fine-tuning compared with standard two-point estimators, highlighting practical impact in high-dimensional or budget-constrained ZOO problems.

Abstract

Zeroth-order optimization (ZOO) is an important framework for stochastic optimization when gradients are unavailable or expensive to compute. A potential limitation of existing ZOO methods is the bias inherent in most gradient estimators unless the perturbation stepsize vanishes. In this paper, we overcome this biasedness issue by proposing a novel family of unbiased gradient estimators based solely on function evaluations. By reformulating directional derivatives as a telescoping series and sampling from carefully designed distributions, we construct estimators that eliminate bias while maintaining favorable variance. We analyze their theoretical properties, derive optimal scaling distributions and perturbation stepsizes of four specific constructions, and prove that SGD using the proposed estimators achieves optimal complexity for smooth non-convex objectives. Experiments on synthetic tasks and language model fine-tuning confirm the superior accuracy and convergence of our approach compared to standard methods.

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization

TL;DR

The paper solves a long-standing bias issue in zeroth-order optimization by constructing unbiased gradient estimators that rely solely on function evaluations. It achieves this through a telescoping-series representation of directional derivatives and a carefully designed sampling scheme over perturbation indices, yielding four concrete estimators that trade off function-query cost and variance. The authors prove unbiasedness, characterize variance bounds, and identify optimal choices for perturbation sequences and sampling distributions, showing that SGD with these estimators attains the optimal complexity for smooth non-convex objectives. Empirically, the approach yields lower gradient-estimation error and faster convergence in synthetic tasks and language-model fine-tuning compared with standard two-point estimators, highlighting practical impact in high-dimensional or budget-constrained ZOO problems.

Abstract

Zeroth-order optimization (ZOO) is an important framework for stochastic optimization when gradients are unavailable or expensive to compute. A potential limitation of existing ZOO methods is the bias inherent in most gradient estimators unless the perturbation stepsize vanishes. In this paper, we overcome this biasedness issue by proposing a novel family of unbiased gradient estimators based solely on function evaluations. By reformulating directional derivatives as a telescoping series and sampling from carefully designed distributions, we construct estimators that eliminate bias while maintaining favorable variance. We analyze their theoretical properties, derive optimal scaling distributions and perturbation stepsizes of four specific constructions, and prove that SGD using the proposed estimators achieves optimal complexity for smooth non-convex objectives. Experiments on synthetic tasks and language model fine-tuning confirm the superior accuracy and convergence of our approach compared to standard methods.
Paper Structure (53 sections, 18 theorems, 90 equations, 3 figures, 1 table)

This paper contains 53 sections, 18 theorems, 90 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Derivation of Unbiased Zeroth-Order Estimators
  3. Telescoping Series and Expectation Representation
  4. On the Role of Absolute Convergence.
  5. The Construction of Unbiased Estimators
  6. Specific Constructions
  7. $\mathsf{P}_4$-Estimator.
  8. $\mathsf{P}_3$-Estimator.
  9. $\mathsf{P}_1$- & $\mathsf{P}_2$-Estimator.
  10. Variance Analysis of Unbiased Zeroth-Order Estimators
  11. Theoretical Analysis
  12. Comparison with Existing Literature.
  13. Comparison with the Noisy Oracle Setup
  14. On the Optimal Choices of $\{ \mu_n\}_{n=1}^\infty$ and $\{ p_n\}_{n=1}^\infty$
  15. Sampling from the Optimal Sampling Distribution $\{ p_n\}_{n=1}^\infty$.
  16. ...and 38 more sections

Key Result

Proposition 2.1

If the second-order continuously differentiable function $f:\mathbb{R}^d \to \mathbb{R}$ has $L$-Lipschitz continuous gradient and $\sum_{n=1}^\infty \mu_n < \infty$, then the series is absolutely convergent and its limit is $\nabla_v f(x)$.

Figures (3)

  • Figure 1: This figure presents the MSE error of four different estimators across various dimensions $d$ ranging from $16$ to $4096$. The left panel corresponds to the quadratic loss $f_{\text{reg}}$, while the right panel illustrates results for the logistic loss $f_{\text{cls}}$. Each box plot describes the distribution of the MSE error across $100$ random trials.
  • Figure 2: Comparison of training loss during fine-tuning of OPT-1.3B on SST-2 using different zeroth-order gradient estimators. The right panel rescales iterations by the number of function evaluations. The unbiased Zipf’s $\mathsf{P}_3$-, Zipf’s $\mathsf{P}_4$-, Geometric $\mathsf{P}_3$-, and Geometric $\mathsf{P}_4$-estimators achieve faster convergence under the same number of function evaluations.
  • Figure 3: Comparison to the two-point estimator with $b=1$ under the same setting as \ref{['fig:lm-optimization']}. We also include the performance of the first-order Adam and SGD in the left panel.

Theorems & Definitions (39)

  • Proposition 2.1
  • Definition 2.2
  • Theorem 2.3: Unbiasedness
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • Example 3.3: Geometric $\mathsf{P}_k$-Estimators
  • Example 3.4: Zipf's $\mathsf{P}_k$-Estimators
  • Corollary 3.5: khaled2022better
  • Definition B.1: Convergent series
  • ...and 29 more