Minimisation of Submodular Functions Using Gaussian Zeroth-Order Random Oracles
Amir Ali Farzin, Yuen-Man Pun, Philipp Braun, Tyler Summers, Iman Shames
TL;DR
The paper tackles submodular function minimisation (SFM) by applying a derivative-free Gaussian smoothing approach to the Lovász extension, enabling zeroth-order optimisation of a convex surrogate. It proves offline convergence to an $ε$-optimal solution in $O(n^2 ε^{-2})$ function calls and establishes online guarantees: static regret is Hannan-consistent and dynamic regret scales as $O\left(\sqrt{N P_N^*}\right)$, where $P_N^*$ is the path length of minimisers. The framework relies on a two-point gradient estimator to unbiasedly estimate the smoothed gradient and includes complexity analysis and hyperparameter guidelines, complemented by numerical demonstrations. The results show the method is competitive with gradient-based approaches in offline and online SFM settings, while offering a practical derivative-free alternative. Potential impact includes scalable, data-driven SFM in domains such as clustering and structured prediction, with extensions to other Lovász-based surrogates and minimax problems.
Abstract
We consider the minimisation problem of submodular functions and investigate the application of a zeroth-order method to this problem. The method is based on exploiting a Gaussian smoothing random oracle to estimate the smoothed function gradient. We prove the convergence of the algorithm to a global $ε$-approximate solution in the offline case and show that the algorithm is Hannan-consistent in the online case with respect to static regret. Moreover, we show that the algorithm achieves $O(\sqrt{NP_N^\ast})$ dynamic regret, where $N$ is the number of iterations and $P_N^\ast$ is the path length. The complexity analysis and hyperparameter selection are presented for all the cases. The theoretical results are illustrated via numerical examples.