Non-asymptotic analysis of the performance of the penalized least trimmed squares in sparse models

TL;DR

This work develops non-asymptotic, finite-sample error bounds for penalized least trimmed squares (LTS) in sparse, high-dimensional regression. It introduces the penalized LTS estimator $\widehat{\boldsymbol{\beta}}^n_{lts-enet}$ with trimming level $h$ and penalties $\lambda_1$, $\lambda_2$, and $\gamma$, and proves existence/uniqueness of the estimator. The main results establish high-probability prediction bounds (Theorem 3.1) and, under sparsity and incoherence conditions, estimation bounds (Theorem 3.2) for fixed design and sub-Gaussian noise, highlighting robustness advantages over standard Lasso-type methods. These finite-sample insights fill a gap for non-convex LTS-based methods in sparse settings and provide a foundation for robust, finite-sample inference in practical, data-limited applications.

Abstract

The least trimmed squares (LTS) estimator is a renowned robust alternative to the classic least squares estimator and is popular in location, regression, machine learning, and AI literature. Many studies exist on LTS, including its robustness, computation algorithms, extension to non-linear cases, asymptotics, etc. The LTS has been applied in the penalized regression in a high-dimensional real-data sparse-model setting where dimension $p$ (in thousands) is much larger than sample size $n$ (in tens, or hundreds). In such a practical setting, the sample size $n$ often is the count of sub-population that has a special attribute (e.g. the count of patients of Alzheimer's, Parkinson's, Leukemia, or ALS, etc.) among a population with a finite fixed size N. Asymptotic analysis assuming that $n$ tends to infinity is not practically convincing and legitimate in such a scenario. A non-asymptotic or finite sample analysis will be more desirable and feasible. This article establishes some finite sample (non-asymptotic) error bounds for estimating and predicting based on LTS with high probability for the first time.