Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Signed-Perturbed Sums Estimation of ARX Systems: Exact Coverage and Strong Consistency (Extended Version)

Algo Carè, Erik Weyer, Balázs Cs. Csáji, Marco C. Campi

TL;DR

The paper extends Sign-Perturbed Sums (SPS) to autoregressive with exogenous input (ARX) models to construct finite-sample confidence regions with exact coverage probability $p=1-q/m$ for any data size. It proves strong consistency (regions shrink to the true parameter $ heta^*$ almost surely) and derives an asymptotic-shape result showing the SPS region is contained in a marginally inflated ellipsoid from standard system identification theory. The ARX adaptation relies on regenerating regressors and outputs under perturbed noise to align distributions and enable a rank-based inclusion test, with demonstrations via simulation that finite-sample SPS regions resemble their asymptotic counterparts while retaining exact finite-sample guarantees. The work provides distribution-free, non-asymptotic confidence guarantees for ARX identification and discusses computational avenues and open questions, including ellipsoidal outer approximations and extensions to broader modeling settings.

Abstract

Sign-Perturbed Sums (SPS) is a system identification method that constructs confidence regions for the unknown system parameters. In this paper, we study SPS for ARX systems, and establish that the confidence regions are guaranteed to include the true model parameter with exact, user-chosen, probability under mild statistical assumptions, a property that holds true for any finite number of observed input-output data. Furthermore, we prove the strong consistency of the method, that is, as the number of data points increases, the confidence region gets smaller and smaller and will asymptotically almost surely exclude any parameter value different from the true one. In addition, we also show that, asymptotically, the SPS region is included in an ellipsoid which is marginally larger than the confidence ellipsoid obtained from the asymptotic theory of system identification. The results are theoretically proven and illustrated in a simulation example.

Signed-Perturbed Sums Estimation of ARX Systems: Exact Coverage and Strong Consistency (Extended Version)

TL;DR

The paper extends Sign-Perturbed Sums (SPS) to autoregressive with exogenous input (ARX) models to construct finite-sample confidence regions with exact coverage probability for any data size. It proves strong consistency (regions shrink to the true parameter almost surely) and derives an asymptotic-shape result showing the SPS region is contained in a marginally inflated ellipsoid from standard system identification theory. The ARX adaptation relies on regenerating regressors and outputs under perturbed noise to align distributions and enable a rank-based inclusion test, with demonstrations via simulation that finite-sample SPS regions resemble their asymptotic counterparts while retaining exact finite-sample guarantees. The work provides distribution-free, non-asymptotic confidence guarantees for ARX identification and discusses computational avenues and open questions, including ellipsoidal outer approximations and extensions to broader modeling settings.

Abstract

Sign-Perturbed Sums (SPS) is a system identification method that constructs confidence regions for the unknown system parameters. In this paper, we study SPS for ARX systems, and establish that the confidence regions are guaranteed to include the true model parameter with exact, user-chosen, probability under mild statistical assumptions, a property that holds true for any finite number of observed input-output data. Furthermore, we prove the strong consistency of the method, that is, as the number of data points increases, the confidence region gets smaller and smaller and will asymptotically almost surely exclude any parameter value different from the true one. In addition, we also show that, asymptotically, the SPS region is included in an ellipsoid which is marginally larger than the confidence ellipsoid obtained from the asymptotic theory of system identification. The results are theoretically proven and illustrated in a simulation example.
Paper Structure (32 sections, 23 theorems, 133 equations, 3 figures, 2 tables)

This paper contains 32 sections, 23 theorems, 133 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem Setting
  3. Data generating system and problem formulation
  4. Least-squares estimate (LSE)
  5. Construction of an Exact Confidence Region
  6. SPS when the regressors are independent of the noise
  7. Main idea behind SPS for ARX systems
  8. SPS for ARX systems
  9. Exact Confidence
  10. Asymptotic Results
  11. Assumptions
  12. Extra assumption for the asymptotic shape
  13. Strong consistency
  14. Asymptotic shape
  15. Confidence ellipsoids based on the asymptotic system identification theory
  16. ...and 17 more sections

Key Result

Theorem 1

Under Assumptions ass:knownorders and ass:NtbasicANDUtbasic, the confidence region constructed by the SPS algorithm in Table inittab and indtab has the property that $\mathbb Pr\{\theta^\ast\in \widehat{\Theta}_n\}=1-\frac{q}{m}$

Figures (3)

  • Figure 1: $95$% confidence regions, $n=40$, $m=100$.
  • Figure 2: $95$% confidence regions, $n=400$, $m=100$.
  • Figure 3: $95$% confidence regions, $n=4000$, $m=4000$ and $m=100$.

Theorems & Definitions (32)

  • Theorem 1
  • Theorem 2: Strong Consistency
  • Theorem 3: Asymptotic Shape
  • Definition 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • proof
  • ...and 22 more