$γ$-weakly $θ$-up-concavity: Linearizable Non-Convex Optimization with Applications to DR-Submodular and OSS Functions
Mohammad Pedramfar, Vaneet Aggarwal
TL;DR
This work addresses the challenge of optimizing monotone non-convex objectives over convex sets by introducing the $\gamma$-weakly $\theta$-up-concave class, which unifies and extends DR-submodular and OSS models. The authors prove upper-linearizability for this entire class, constructing a surrogate linearization that yields approximation guarantees dependent only on $\gamma$, $\theta$, and the feasible-region geometry. Leveraging the upper-linearizable framework, they obtain offline and online guarantees, including improved constants under matroid constraints and broad applicability to various feedback models via meta-algorithms that transfer linear-optimization results to nonlinear objectives. They also instantiate this framework for the $(p,\sigma)$-up-concave family, deriving explicit coefficients such as $1-\exp(-\gamma/(\sigma+1))$ for matroids and $1-\exp(-1/(2\sigma+1))$ for OSS, while avoiding higher-order smoothness assumptions. The resulting reductions yield projection-free online algorithms with favorable regret bounds and sample complexities, plus robust offline guarantees, expanding the practical reach of efficient optimization for a wide class of monotone nonconcave functions.
Abstract
Optimizing monotone non-convex functions is a fundamental challenge across machine learning and combinatorial optimization. We introduce and study $γ$-weakly $θ$-up-concavity, a novel first-order condition that characterizes a broad class of such functions. This condition provides a powerful unifying framework, strictly generalizing both DR-submodular functions and One-Sided Smooth (OSS) functions. Our central theoretical contribution demonstrates that $γ$-weakly $θ$-up-concave functions are upper-linearizable: for any feasible point, we can construct a linear surrogate whose gains provably approximate the original non-linear objective. This approximation holds up to a constant factor, namely the approximation coefficient, dependent solely on $γ$, $θ$, and the geometry of the feasible set. This linearizability yields immediate and unified approximation guarantees for a wide range of problems. Specifically, we obtain unified approximation guarantees for offline optimization as well as static and dynamic regret bounds in online settings via standard reductions to linear optimization. Moreover, our framework recovers the optimal approximation coefficient for DR-submodular maximization and significantly improves existing approximation coefficients for OSS optimization, particularly over matroid constraints.