Robust Least Squares Problems with Binary Uncertain Data
Yang Zhou, Xiaojun Chen
TL;DR
This work tackles robust least squares with binary or hypercube adversarial noise by formulating a minimax problem $\min_{x \in \mathcal{X}} \max_{y \in \mathcal{Y}} \Theta(x,y)$ where $\Theta(x,y)=\tfrac{1}{2}\|F(x)-C y\|^2$. It reveals that the geometry of the noise-propagation matrix $C$—whether its columns form acute or obtuse angles—determines whether the inner maximization is supermodular or submodular, enabling submodular optimization tools within a gradient-based minimax framework. For the supermodular linear case, a Lovász-extension-based approach relates global minimax points to saddle points, yielding an $\epsilon$-global minimax point in $O(\epsilon^{-2})$ iterations; for the nonlinear case, a randomized projected-gradient method achieves $\epsilon$-stationarity in $O(\epsilon^{-4})$. In the submodular linear case, a double greedy subsolver provides a $(\tfrac{1}{3},\epsilon)$-approximate minimax point in $O(\epsilon^{-2})$, with exactness when $C$ is column-orthogonal. Numerical experiments on health status prediction and phase retrieval demonstrate BRLS's superior robustness to structured noise compared with classical LS and LASSO, highlighting its practical impact in adversarially noisy settings.
Abstract
We propose a Binary Robust Least Squares (BRLS) model that encompasses key robust least squares formulations, such as those involving uncertain binary labels and adversarial noise constrained within a hypercube. We show that the geometric structure of the noise propagation matrix, particularly whether its columns form acute or obtuse angles, implies the supermodularity or submodularity of the inner maximization problem. This structural property enables us to integrate powerful combinatorial optimization tools into a gradient-based minimax algorithmic framework. For the robust linear least squares problem with the supermodularity, we establish the relationship between the minimax points of BRLS and saddle points of its continuous relaxation, and propose a projected gradient algorithm computing $ε$-global minimax points in $O(ε^{-2})$ iterations. For the robust nonlinear least squares problem with supermodularity, we develop a revised framework that finds $ε$-stationary points in the sense of expectation within $O(ε^{-4})$ iterations. For the robust linear least squares problem with the submodularity, we employ a double greedy algorithm as a subsolver, guaranteeing a $(\frac{1}{3}, ε)$-approximate minimax point in $O(ε^{-2})$ iterations. Numerical experiments on health status prediction and phase retrieval demonstrate that BRLS achieves superior robustness against structured noise compared to classical least squares problems and LASSO.