Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums

Szabolcs Szentpéteri, Balázs Csanád Csáji

TL;DR

This work introduces Residual-Permuted Sums (RPS), a distribution-free, finite-sample method for constructing confidence regions in linear regression by permuting residuals rather than perturbing signs. Under mild, general assumptions including i.i.d. noises $W_t$ with finite fourth moments, RPS achieves exact finite-sample coverage with a provable uniform strong consistency, and it extends the existing Sign-Perturbed Sums (SPS) ellipsoidal outer-approximation (EOA) to the permutation-based setting. A permutation-lemma-based proof establishes a strong law for randomly permuted sequences, which underpins the uniform convergence of RPS regions to the true parameter $\theta^*$. Simulation results demonstrate that RPS regions can be tighter than SPS and comparable to asymptotic regions while preserving finite-sample guarantees, highlighting its practical value for robust system identification and related applications.

Abstract

This letter studies a distribution-free, finite-sample data perturbation (DP) method, the Residual-Permuted Sums (RPS), which is an alternative of the Sign-Perturbed Sums (SPS) algorithm, to construct confidence regions. While SPS assumes independent (but potentially time-varying) noise terms which are symmetric about zero, RPS gets rid of the symmetricity assumption, but assumes i.i.d. noises. The main idea is that RPS permutes the residuals instead of perturbing their signs. This letter introduces RPS in a flexible way, which allows various design-choices. RPS has exact finite sample coverage probabilities and we provide the first proof that these permutation-based confidence regions are uniformly strongly consistent under general assumptions. This means that the RPS regions almost surely shrink around the true parameters as the sample size increases. The ellipsoidal outer-approximation (EOA) of SPS is also extended to RPS, and the effectiveness of RPS is validated by numerical experiments, as well.

Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums

TL;DR

This work introduces Residual-Permuted Sums (RPS), a distribution-free, finite-sample method for constructing confidence regions in linear regression by permuting residuals rather than perturbing signs. Under mild, general assumptions including i.i.d. noises with finite fourth moments, RPS achieves exact finite-sample coverage with a provable uniform strong consistency, and it extends the existing Sign-Perturbed Sums (SPS) ellipsoidal outer-approximation (EOA) to the permutation-based setting. A permutation-lemma-based proof establishes a strong law for randomly permuted sequences, which underpins the uniform convergence of RPS regions to the true parameter . Simulation results demonstrate that RPS regions can be tighter than SPS and comparable to asymptotic regions while preserving finite-sample guarantees, highlighting its practical value for robust system identification and related applications.

Abstract

This letter studies a distribution-free, finite-sample data perturbation (DP) method, the Residual-Permuted Sums (RPS), which is an alternative of the Sign-Perturbed Sums (SPS) algorithm, to construct confidence regions. While SPS assumes independent (but potentially time-varying) noise terms which are symmetric about zero, RPS gets rid of the symmetricity assumption, but assumes i.i.d. noises. The main idea is that RPS permutes the residuals instead of perturbing their signs. This letter introduces RPS in a flexible way, which allows various design-choices. RPS has exact finite sample coverage probabilities and we provide the first proof that these permutation-based confidence regions are uniformly strongly consistent under general assumptions. This means that the RPS regions almost surely shrink around the true parameters as the sample size increases. The ellipsoidal outer-approximation (EOA) of SPS is also extended to RPS, and the effectiveness of RPS is validated by numerical experiments, as well.
Paper Structure (13 sections, 3 theorems, 41 equations, 2 figures, 2 algorithms)

This paper contains 13 sections, 3 theorems, 41 equations, 2 figures, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Problem Setting
  3. Data Generation
  4. Assumptions
  5. Co-regressor construction
  6. The Residual-Permuted Sums Algorithm
  7. Exact coverage of RPS confidence regions
  8. Strong Consistency of RPS Regions
  9. Permutation lemma
  10. Strong consistency
  11. Ellipsoidal Outer-Approximation
  12. Simulation Experiments
  13. Conclusions

Key Result

Theorem 1

Assuming $\{W_t\}$ are i.i.d., and $R_n$, $\{\psi_t\}$ are independent of $\{W_t\}$, the coverage probability of the constructed confidence region $\mathcal{C}_{p,n}$ is exactly $p$, that is

Figures (2)

  • Figure 1: Comparison of 0.9-level RPS indicator, RPS EOA, SPS indicator, SPS EOA and asymptotic confidence regions for $n=250$.
  • Figure 2: Comparison of 0.9-level RPS indicator and asymptotic confidence regions for $n=200$, $n=1000$ and $n=2000$.

Theorems & Definitions (6)

  • Theorem 1
  • proof
  • Lemma 1
  • proof
  • Theorem 2
  • proof