Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model selection by cross-validation in an expectile linear regression

Bilel Bousselmi, Gabriela Ciuperca

TL;DR

This paper addresses model selection in linear expectile regression under asymmetric errors by proposing a cross-validation criterion (CVS) that leverages training-set expectile estimators and large validation sets. It establishes model-selection consistency for both fixed and high-dimensional regimes, including two growth-rate subregimes for $p_n$ with detailed rate conditions and assumptions. Through simulations, the authors show that CVS-based expectile selection often outperforms CVS-based quantile or least-squares approaches, especially under strong skewness, and demonstrate practical usefulness with real-data applications in aquatic toxicity and energy consumption. The results highlight the robustness and scalability of the CV expectile framework for reliable variable selection without relying on normality, and its advantage in identifying irrelevant variables directly.

Abstract

For linear models that may have asymmetric errors, we study variable selection by cross-validation. The data are split into training and validation sets, with the number of observations in the validation set much larger than in the training set. For the model coefficients, the expectile or adaptive LASSO expectile estimators are calculated on the training set. These estimators will be used to calculate the cross-validation mean score (CVS) on the validation set. We show that the model that minimizes CVS is consistent in two cases: when the number of explanatory variables is fixed or when it depends on the number of observations. Monte Carlo simulations confirm the theoretical results and demonstrate the superiority of our estimation method compared to two others in the literature. The usefulness of the CV expectile model selection technique is illustrated by applying it to real data sets.

Model selection by cross-validation in an expectile linear regression

TL;DR

This paper addresses model selection in linear expectile regression under asymmetric errors by proposing a cross-validation criterion (CVS) that leverages training-set expectile estimators and large validation sets. It establishes model-selection consistency for both fixed and high-dimensional regimes, including two growth-rate subregimes for with detailed rate conditions and assumptions. Through simulations, the authors show that CVS-based expectile selection often outperforms CVS-based quantile or least-squares approaches, especially under strong skewness, and demonstrate practical usefulness with real-data applications in aquatic toxicity and energy consumption. The results highlight the robustness and scalability of the CV expectile framework for reliable variable selection without relying on normality, and its advantage in identifying irrelevant variables directly.

Abstract

For linear models that may have asymmetric errors, we study variable selection by cross-validation. The data are split into training and validation sets, with the number of observations in the validation set much larger than in the training set. For the model coefficients, the expectile or adaptive LASSO expectile estimators are calculated on the training set. These estimators will be used to calculate the cross-validation mean score (CVS) on the validation set. We show that the model that minimizes CVS is consistent in two cases: when the number of explanatory variables is fixed or when it depends on the number of observations. Monte Carlo simulations confirm the theoretical results and demonstrate the superiority of our estimation method compared to two others in the literature. The usefulness of the CV expectile model selection technique is illustrated by applying it to real data sets.
Paper Structure (13 sections, 3 theorems, 75 equations, 3 figures, 10 tables)

This paper contains 13 sections, 3 theorems, 75 equations, 3 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Model and methods
  3. Asymptotic behavior of CV model estimators
  4. Case of fixed $p$
  5. Case $p=p_n$, $p_n=O(n_{\cal A}^c)$
  6. Case $0<c<1/2$
  7. Case $1/2\leq c \leq 1$
  8. Simulation studies
  9. Application on real data
  10. Acute aquatic toxicity
  11. Energy Consumption
  12. Conclusion and CV expectile method strengths
  13. Proofs of main results

Key Result

Theorem 3.1

Under assumptions (A1), (A2), (A3), (A4), we have: $\mathbb{P}[\widehat{\cal M}_n ={\cal M}^*]{\underset{n \rightarrow \infty}{\longrightarrow}} 1$.

Figures (3)

  • Figure 1: Histograms and density estimations for four distributions of $\varepsilon$.
  • Figure 2: Acute aquatic toxicity data: standardized residuals corresponding to the CV expectile method on the training and validation sets.
  • Figure 3: Energy Consumption data: standardized residuals corresponding to the CV expectile method on the training and validation sets.

Theorems & Definitions (5)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Remark 3.1
  • Remark 3.2