Robust Solutions of Nonlinear Least Squares Problems via Min-max Optimization

TL;DR

The paper addresses robustness in nonlinear least squares by formulating a min-max problem $\min_x \max_{y \in \Omega} f(x,y)$ with $f(x,y)=\|F(x)-C y\|^2$ and $\Omega=\{y: \|y\|_\infty \le \delta\}$, derives an explicit inner value function $\varphi(x)$ and proves the existence of global minimax points. A smoothing-based approach is developed to compute minimax points with complexity guarantees that scale as $O(\varepsilon^{-2})$ for the smooth part and $O(\varepsilon^{-2}+\delta^2 \varepsilon^{-3})$ for the full min-max problem, exploiting a structured QR factorization of the perturbation matrix. The authors provide error bounds linking standard nonlinear LS solutions to robust LS solutions, and demonstrate the method on integral-equation problems with uncertain data to show improved robustness over naive LS. The results offer practical procedures for robust LS in science and engineering, with clear computational complexity and reproducibility resources.

Abstract

This paper considers robust solutions to a class of nonlinear least squares problems using min-max optimization approach. We give an explicit formula for the value function of the inner maximization problem and show the existence of global minimax points. We establish error bounds from any solution of the nonlinear least squares problem to the solution set of the robust nonlinear least squares problem. Moreover, we propose a smoothing method for finding a global minimax point of the min-max problem by using the formula and show that finding an $ε$ minimax critical point of the min-max problem needs at most $O(ε^{-2} +δ^2 ε^{-3})$ evaluations of the function value and gradients of the objective function, where $δ$ is the tolerance of the noise. Numerical results of integral equations with uncertain data demonstrate the robustness of solutions of our approach and unstable behaviour of least squares solutions disregarding uncertainties in the data.

Figures (3)

• Figure 1: Nonlinear linear least squares example, $\Delta_\lambda(\delta)=Er_\delta(\vp_{ls}) - Er_\delta(\vp_{rls}^\lambda)$
• Figure 2: Linear least squares example, $\Delta_\lambda(\delta)=Er_\delta(\vp_{ls}) - Er_\delta(\vp_{rls}^\lambda)$
• Figure 3: Robust linear least squares vs Lasso, $\Delta_\lambda(\delta)=Er_\delta(\vp^\lambda_{las}) - Er_\delta(\vp_{rls}^\lambda)$

Theorems & Definitions (22)

• theorem 2.1
• proof
• theorem 2.2
• proof
• proposition 1
• proof
• theorem 3.1
• proof
• proposition 2
• proof