Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic Optimization Algorithms for Problems with Controllable Biased Oracles

Yin Liu, Sam Davanloo Tajbakhsh

TL;DR

The paper tackles optimization with controllable gradient bias, introducing a bias-control framework where the bias magnitude $h_b(ta)$ decreases as $ta$ grows and incurs additional computation. It develops adaptive and variance-reduced biased algorithms—AB-SG, AB-VSG, and their proximal/multistage variants—and provides nonasymptotic guarantees in nonconvex settings, quantified by traditional sample complexity and new $ta$- and $ta B$-complexities. The results show power-law decay of bias in composition optimization and exponential decay in infinite-horizon MDPs, with concrete rates such as $ ilde{O}(psilon^{-4})$ (baseline) and $ ilde{O}(psilon^{-3})$ (variance-reduced), along with high-probability bounds. Empirical validation across composition optimization, MDP policy optimization, and distributionally robust optimization demonstrates practical gains from adaptive bias control, confirming the framework's relevance for broad stochastic optimization problems where unbiased gradients are costly or unavailable.

Abstract

Motivated by emerging applications in machine learning, we consider an optimization problem in a general form where the gradient of the objective function is available through a biased stochastic oracle. We assume a bias-control parameter can reduce the bias magnitude; however, a lower bias requires more computation/samples. For instance, in two applications on stochastic composition optimization and policy optimization for infinite-horizon Markov decision processes, we show that the bias follows a power law and exponential decay, respectively, as functions of their corresponding bias control parameters. For problems with such gradient oracles, the paper proposes stochastic algorithms that adjust the bias-control parameter throughout the iterations. We analyze the nonasymptotic performance of the proposed algorithms in the nonconvex regime and establish their sample or bias-control computational complexities to obtain a stationary point in expectation or with high probability. Finally, we numerically evaluate the performance of the proposed algorithms over three applications.

Stochastic Optimization Algorithms for Problems with Controllable Biased Oracles

TL;DR

The paper tackles optimization with controllable gradient bias, introducing a bias-control framework where the bias magnitude decreases as grows and incurs additional computation. It develops adaptive and variance-reduced biased algorithms—AB-SG, AB-VSG, and their proximal/multistage variants—and provides nonasymptotic guarantees in nonconvex settings, quantified by traditional sample complexity and new - and -complexities. The results show power-law decay of bias in composition optimization and exponential decay in infinite-horizon MDPs, with concrete rates such as (baseline) and (variance-reduced), along with high-probability bounds. Empirical validation across composition optimization, MDP policy optimization, and distributionally robust optimization demonstrates practical gains from adaptive bias control, confirming the framework's relevance for broad stochastic optimization problems where unbiased gradients are costly or unavailable.

Abstract

Motivated by emerging applications in machine learning, we consider an optimization problem in a general form where the gradient of the objective function is available through a biased stochastic oracle. We assume a bias-control parameter can reduce the bias magnitude; however, a lower bias requires more computation/samples. For instance, in two applications on stochastic composition optimization and policy optimization for infinite-horizon Markov decision processes, we show that the bias follows a power law and exponential decay, respectively, as functions of their corresponding bias control parameters. For problems with such gradient oracles, the paper proposes stochastic algorithms that adjust the bias-control parameter throughout the iterations. We analyze the nonasymptotic performance of the proposed algorithms in the nonconvex regime and establish their sample or bias-control computational complexities to obtain a stationary point in expectation or with high probability. Finally, we numerically evaluate the performance of the proposed algorithms over three applications.
Paper Structure (26 sections, 24 theorems, 110 equations, 8 figures, 1 table, 5 algorithms)

This paper contains 26 sections, 24 theorems, 110 equations, 8 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Motivating examples
  3. Related works
  4. Biased stochastic gradient descent (B-SGD) with uncontrollable bias
  5. Biased stochastic algorithms with controllable bias
  6. Contributions
  7. Preliminaries and notations
  8. Assumptions
  9. Smooth regime with biased stochastic oracle
  10. Biased stochastic gradient descent
  11. Adaptively-biased stochastic gradient method
  12. Adaptively-biased variance-reduced stochastic gradient method
  13. Nonsmooth regime with biased oracle of the smooth component
  14. Proximal biased stochastic gradient method
  15. Multistage proximal biased variance-reduced stochastic gradient method
  16. ...and 11 more sections

Key Result

Lemma 1.1

Assume the following inequalities hold, $\mathbb{E}_\varphi[\left\|\nabla f_\varphi(\mathbf{u}) - \nabla f_\varphi(\mathbf{u}')\right\|^2]\leq L_f^2\left\|\mathbf{u}-\mathbf{u}'\right\|^2$, $\mathbb{E}_\xi[\left\|g_\xi(\mathbf{x})-g(\mathbf{x})\right\|^2]\leq \sigma_g^2$, and $\mathbb{E}_\xi [\left\ When $\mathcal{B}_g$ and $\mathcal{B}_{\nabla g}$ are independent of each other, the bias is bounde

Figures (8)

  • Figure 1: Estimating $h_b(\eta)$ and $h_v(\eta)$ for the composition optimization problem
  • Figure 2: Performance of three algorithms for the composition optimization problem
  • Figure 3: Comparison of the $\eta$-complexity of the three algorithms in the composition optimization problem
  • Figure 4: Estimating $h_b(\eta)$ and $h_v(\eta)$ through simulation for the infinite-horizon MDP
  • Figure 5: Performance of three algorithms for the infinite-horizon MDP problem
  • ...and 3 more figures

Theorems & Definitions (44)

  • Example 1: Composition optimization
  • Lemma 1.1
  • Example 2: Infinite-horizon Markov decision process
  • Lemma 1.2
  • Example 3: Distributionally robust optimization
  • Remark 1.3
  • Remark 2.3
  • Theorem 3.1
  • Remark 3.2
  • Corollary 3.3
  • ...and 34 more