Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unbiased Single-Queried Gradient for Combinatorial Objective

Thanawat Sornwanee

TL;DR

This paper tackles optimizing combinatorial objectives defined on binary vectors by relaxing to a product-Bernoulli multilinear extension $v(x)$ and seeking gradients in the continuous space. It introduces Easy Stochastic Gradient (ESG), an unbiased single-queried gradient estimator that yields $\mathbb{E}[\nabla_x V(x;\omega,Q)]=\nabla_x v(x;Q)$ while using only one oracle call per realization, enabling gradient-based methods with autodiff. ESG relies on a carefully designed stochastic evaluation $V$ built from a good tuple $(f,\sigma,\hat{σ})$ and a per-coordinate factorization, unifying ties to REINFORCE through importance sampling and generating new estimators (e.g., Long Jump, Arch, Spike) with variance considerations. The paper further develops Calibration, Variance control, and a Stochastic Query Descent (SQD) framework, showing through symmetric-slice and encoded variants that one-query, unbiased, differentiable optimization is viable for large-scale combinatorial problems. The encoded ESG/EncodedSQD variants illustrate practical gains in computation and variance, highlighting the approach’s potential for scalable, black-box combinatorial optimization where oracle calls dominate cost.

Abstract

In a probabilistic reformulation of a combinatorial problem, we often face an optimization over a hypercube, which corresponds to the Bernoulli probability parameter for each binary variable in the primal problem. The combinatorial nature suggests that an exact gradient computation requires multiple queries. We propose a stochastic gradient that is unbiased and requires only a single query of the combinatorial function. This method encompasses a well-established REINFORCE (through an importance sampling), as well as including a class of new stochastic gradients.

Unbiased Single-Queried Gradient for Combinatorial Objective

TL;DR

This paper tackles optimizing combinatorial objectives defined on binary vectors by relaxing to a product-Bernoulli multilinear extension and seeking gradients in the continuous space. It introduces Easy Stochastic Gradient (ESG), an unbiased single-queried gradient estimator that yields while using only one oracle call per realization, enabling gradient-based methods with autodiff. ESG relies on a carefully designed stochastic evaluation built from a good tuple and a per-coordinate factorization, unifying ties to REINFORCE through importance sampling and generating new estimators (e.g., Long Jump, Arch, Spike) with variance considerations. The paper further develops Calibration, Variance control, and a Stochastic Query Descent (SQD) framework, showing through symmetric-slice and encoded variants that one-query, unbiased, differentiable optimization is viable for large-scale combinatorial problems. The encoded ESG/EncodedSQD variants illustrate practical gains in computation and variance, highlighting the approach’s potential for scalable, black-box combinatorial optimization where oracle calls dominate cost.

Abstract

In a probabilistic reformulation of a combinatorial problem, we often face an optimization over a hypercube, which corresponds to the Bernoulli probability parameter for each binary variable in the primal problem. The combinatorial nature suggests that an exact gradient computation requires multiple queries. We propose a stochastic gradient that is unbiased and requires only a single query of the combinatorial function. This method encompasses a well-established REINFORCE (through an importance sampling), as well as including a class of new stochastic gradients.
Paper Structure (31 sections, 18 theorems, 102 equations, 6 figures, 4 algorithms)

This paper contains 31 sections, 18 theorems, 102 equations, 6 figures, 4 algorithms.

Table of Contents

  1. Introduction
  2. Why the obvious approach fails.
  3. Related Work
  4. Score-function estimators and variance reduction
  5. Continuous relaxations and straight-through gradients
  6. Zeroth-order / black-box gradient methods
  7. Stochastic Gradient
  8. Easy Stochastic Gradient
  9. Examples
  10. Calibration
  11. Easy Stochastic Gradient is not Calibrated
  12. Variance, Calibration, & Importance Sampling
  13. Stochastic Query Descent
  14. Experiment: Symmetric-Slice Optimization
  15. Preliminary Result
  16. ...and 16 more sections

Key Result

Proposition 3.3

With respect to $\left( \Omega, \mathcal{F}, \mathbb{P} \right)$, $V$ is a stochastic evaluation function w with $f_v$ and $K$ as in the definition def:seval if and only if, for all $q \in \mathbb{R}$, $x \in (0,1)^d$, and $y \in \{0,1\}^d$,

Figures (6)

  • Figure 1: Computational graph for our ESG algorithm \ref{['alg:ESG']}: The yellow arrows represent operations that allow backpropagation of gradient, while the red one, which requires either rounding or querying, does not. The stochastic value $v$ itself can be taken gradient with respect to $x$, while yielding an unbiased gradient estimator.
  • Figure 2: Median of the best-so-far progress of different algorithms with inter-quartiles uncertainty. The algorithm we proposed are in bold lines, and the pre-existing algorithms are in dashed lines. This result comes from the case when the dimension $d= 30$.
  • Figure 3: Result for the experiment for symmetric-slice optimization from \ref{['subsection:symmetricslice']}: Median of the best-so-far progress of different algorithms with inter-quartiles uncertainty. The algorithm we proposed are in bold line, and the pre-existing algorithms are in a dashed line. The experiment comes from $20$ trials. Note that the time-scale for the 3 plots (with 3 different dimensions) are different.
  • Figure 4: Result for the experiment for symmetric-slice optimization from \ref{['subsection:symmetricslice']}: The bold lines are our SQD algorithms, while the dashed lines are our Encoded SQD algorithms. The encoding function $\hat{\sigma}^{-1}\left( \cdot \right)$ is linear in the case of "SQD: LongJump", so the two methods, "SQD: LongJump" and 'EncodedSQD: LongJump" have roughly the same performance.
  • Figure 5: Result for the experiment for modified Knapsack problem: Median of the best-so-far progress of different algorithms with inter-quartiles uncertainty. The algorithm we proposed are in bold line, and the pre-existing algorithms are in a dashed line. The experiment comes from $20$ trials. Note that the time-scale for the 3 plots (with 3 different dimensions) are different.
  • ...and 1 more figures

Theorems & Definitions (53)

  • Definition 3.1
  • Definition 3.2
  • Proposition 3.3
  • Definition 3.4
  • Corollary 3.5
  • proof
  • Remark 3.6: Stochastic Evaluation by Stochastic Key
  • Remark 3.7: REINFROCE Gradient
  • Definition 3.8
  • Theorem 3.9
  • ...and 43 more