Worst-case Nonlinear Regression with Error Bounds

TL;DR

The paper tackles the challenge of learning nonlinear surrogates with guaranteed worst-case error bounds over a compact input set. It introduces a minimax regression framework with a smooth max surrogate and an active-learning loop that queries the global error maximizer to iteratively improve the model. The authors present methods to compute constant and input-dependent error envelopes, as well as constraint-satisfaction techniques, and demonstrate applications to uncertain dynamical models and explicit MPC. The results show that active-learning-focused minimax training can yield robust, certifiable approximations with practical impact for robust control and safety-critical systems.

Abstract

This paper proposes an active-learning approach to worst-case nonlinear regression with deterministic error guarantees. Given a known nonlinear function defined over a compact set, we compute a surrogate model, such as a feedforward neural network, by minimizing the maximum absolute approximation error. To address the nonsmooth nature of the resulting minimax problem, we introduce a smooth approximation of the $L_\infty$-type loss that enables efficient gradient-based training. We iteratively enrich the training set by actively learning points of largest approximation error through global optimization. The resulting models admit certified worst-case error bounds, either constant or input-dependent, over the entire input domain. The approach is demonstrated through approximations of nonlinear functions and nonconvex sets, as well as through the derivation of uncertain models of more complex nonlinear dynamics within a given model class, and the approximation of explicit model predictive control laws.

Figures (8)

• Figure 1: Uncertainty bounds obtained by running Algorithm \ref{['algo:max_error_fit']} on Example \ref{['eq:example-scalar']}.
• Figure 2: Execution of Algorithm \ref{['algo:max_error_fit']} on Example \ref{['eq:example-scalar']}: maximum error $e_N$ (left plot) and samples acquired (right plot).
• Figure 3: Gaussian function. Top-left: function $f(x)$; Top-right: learned leaky-ReLU $\hat{f}(x;\theta)$; Bottom-left: pointwise error $e(x)=f(x)-\hat{f}(x;\theta)$ and envelope $[\bar{e}^\star_{\min}(x),\bar{e}^\star_{\max}(x)]$; Bottom-right: WCE and MSE over active-learning iterations of Algorithm \ref{['algo:max_error_fit']}.
• Figure 4: Convex inner approximation of a nonconvex set ${\mathcal{S}}$ (blue) using ($i$) convex PWA function (left) and ($ii$) input-convex NN (right). Feasible points are in green, infeasible points in orange. The initial training points are shown as boxes, the actively-learned points as stars.
• Figure 5: Pendulum model example: reconstructed discrete-time PWA mapping for $\xi_1(k+1)$ (left) and $\xi_2(k+1)$ (right), and corresponding approximation error for $u=0$ Nm.

Theorems & Definitions (4)

• Proposition 1
• Proposition 2
• Example 1
• Proposition 3