Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Robustness of Interval CVaR Based Regression Models under Perturbation and Contamination

Yulei You, Junyi Liu

TL;DR

This work analyzes robustness of nonlinear regression when using interval CVaR (In-CVaR) as the loss risk measure, addressing both contamination and perturbation. It introduces a distributional breakdown-point (BP) framework and proves lower and upper BP bounds that converge to $\min\{\beta,1-\beta\}$ under broad conditions, with a key finding that qualitative robustness under perturbation requires trimming the largest losses ($\beta<1$). The results apply to linear, piecewise affine, and neural-network regression with $\ell_1$, $\ell_2$, and Huber losses, supported by probabilistic conditions on the nominal distribution and extensive model verification. Numerical experiments on piecewise affine regression corroborate the theory, showing In-CVaR's robustness advantages over CVaR and expectation in both contamination and perturbation regimes, and highlighting practical relevance for robust learning and deep-network deployment.

Abstract

Robustness under perturbation and contamination is a prominent issue in statistical learning. We address the robust nonlinear regression based on the so-called interval conditional value-at-risk (In-CVaR), which is introduced to enhance robustness by trimming extreme losses. While recent literature shows that the In-CVaR based statistical learning exhibits superior robustness performance than classical robust regression models, its theoretical robustness analysis for nonlinear regression remains largely unexplored. We rigorously quantify robustness under contamination, with a unified study of distributional breakdown point for a broad class of regression models, including linear, piecewise affine and neural network models with $\ell_1$, $\ell_2$ and Huber losses. Moreover, we analyze the qualitative robustness of the In-CVaR based estimator under perturbation. We show that under several minor assumptions, the In-CVaR based estimator is qualitatively robust in terms of the Prokhorov metric if and only if the largest portion of losses is trimmed. Overall, this study analyzes robustness properties of In-CVaR based nonlinear regression models under both perturbation and contamination, which illustrates the advantages of In-CVaR risk measure over conditional value-at-risk and expectation for robust regression in both theory and numerical experiments.

Statistical Robustness of Interval CVaR Based Regression Models under Perturbation and Contamination

TL;DR

This work analyzes robustness of nonlinear regression when using interval CVaR (In-CVaR) as the loss risk measure, addressing both contamination and perturbation. It introduces a distributional breakdown-point (BP) framework and proves lower and upper BP bounds that converge to under broad conditions, with a key finding that qualitative robustness under perturbation requires trimming the largest losses (). The results apply to linear, piecewise affine, and neural-network regression with , , and Huber losses, supported by probabilistic conditions on the nominal distribution and extensive model verification. Numerical experiments on piecewise affine regression corroborate the theory, showing In-CVaR's robustness advantages over CVaR and expectation in both contamination and perturbation regimes, and highlighting practical relevance for robust learning and deep-network deployment.

Abstract

Robustness under perturbation and contamination is a prominent issue in statistical learning. We address the robust nonlinear regression based on the so-called interval conditional value-at-risk (In-CVaR), which is introduced to enhance robustness by trimming extreme losses. While recent literature shows that the In-CVaR based statistical learning exhibits superior robustness performance than classical robust regression models, its theoretical robustness analysis for nonlinear regression remains largely unexplored. We rigorously quantify robustness under contamination, with a unified study of distributional breakdown point for a broad class of regression models, including linear, piecewise affine and neural network models with , and Huber losses. Moreover, we analyze the qualitative robustness of the In-CVaR based estimator under perturbation. We show that under several minor assumptions, the In-CVaR based estimator is qualitatively robust in terms of the Prokhorov metric if and only if the largest portion of losses is trimmed. Overall, this study analyzes robustness properties of In-CVaR based nonlinear regression models under both perturbation and contamination, which illustrates the advantages of In-CVaR risk measure over conditional value-at-risk and expectation for robust regression in both theory and numerical experiments.
Paper Structure (30 sections, 17 theorems, 155 equations, 5 figures, 2 tables)

This paper contains 30 sections, 17 theorems, 155 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Distributional BP for In-CVaR Based Regression Models
  3. Notations and Problem Statement
  4. Concepts of Breakdown Point of Statistical Estimation
  5. Lower Bounds of the Distributional BP of In-CVaR Based Estimators
  6. Upper Bounds of the Distributional BP of In-CVaR Based Estimators
  7. Distributional BP of Typical Regression Models
  8. Linear Regression
  9. Piecewise Affine Regression
  10. Qualitative Statistical Robustness of In-CVaR Based Estimators
  11. Consistency of In-CVaR Based Estimator
  12. Sufficient Conditions for Qualitative Robustness
  13. Necessary Conditions for Qualitative Robustness
  14. Numerical Experiments
  15. Distributional BP Results for In-CVaR Based Piecewise Affine Regression
  16. ...and 15 more sections

Key Result

Proposition 1

Suppose that $f(\bullet, \bullet): \mathbb{R}^p \times \mathbb{R}^{p+1} \to \mathbb{R}$ is a linear regression function with $f(\boldsymbol{x}, \BFtheta_1, \theta_0) = \BFtheta_1^\top \boldsymbol{x} + \theta_0$. For the estimator $\hat{\mathcal{S}} (D, f) \subseteq \mathbb{R}^{p+1}$ under the linear

Figures (5)

  • Figure 1: Logical flowchart of the sufficient conditions for qualitative robustness.
  • Figure 2: Estimated regression functions under the varying contamination levels.
  • Figure 3: Effect of levels $(\alpha, \beta)$ of In-CVaR estimator with $\varepsilon = 0.05.$
  • Figure 4: Estimated regression functions under perturbation.
  • Figure 5: Construction in the proof of Proposition \ref{['prop:upper_piecewiseaffine']} for $I = J = 2$, $p = 1$ and $n=1$.

Theorems & Definitions (40)

  • Definition 1: Distributional breakdown point
  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Theorem 1
  • proof
  • Proposition 3
  • proof
  • Theorem 2
  • ...and 30 more