Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Finite-Sample Inference for Sparsely Permuted Linear Regression

Hirofumi Ota, Masaaki Imaizumi

Abstract

We study a linear observation model with an unknown permutation called \textit{permuted/shuffled linear regression}, where responses and covariates are mismatched and the permutation forms a discrete, factorial-size parameter. The permutation is a key component of the data-generating process, yet its statistical investigation remains challenging due to its discrete nature. We develop a general statistical inference framework on the permutation and regression coefficients. First, we introduce a localization step that reduces the permutation space to a small candidate set building on recent advances in the repro samples method, whose miscoverage decays polynomially with the number of Monte Carlo samples. Then, based on this localized set, we provide statistical inference procedures: a conditional Monte Carlo test of permutation structures with valid finite-sample Type-I error control. We also develop coefficient inference that remains valid under alignment uncertainty of permutations. For computational purposes, we develop a linear assignment problem computable in polynomial time and demonstrate that, with high probability, the solution is equivalent to that of the conventional least squares with large computational cost. Extensions to partially permuted designs and ridge regularization are further discussed. Extensive simulations and an application to air-quality data corroborate finite-sample validity, strong power to detect mismatches, and practical scalability.

Finite-Sample Inference for Sparsely Permuted Linear Regression

Abstract

We study a linear observation model with an unknown permutation called \textit{permuted/shuffled linear regression}, where responses and covariates are mismatched and the permutation forms a discrete, factorial-size parameter. The permutation is a key component of the data-generating process, yet its statistical investigation remains challenging due to its discrete nature. We develop a general statistical inference framework on the permutation and regression coefficients. First, we introduce a localization step that reduces the permutation space to a small candidate set building on recent advances in the repro samples method, whose miscoverage decays polynomially with the number of Monte Carlo samples. Then, based on this localized set, we provide statistical inference procedures: a conditional Monte Carlo test of permutation structures with valid finite-sample Type-I error control. We also develop coefficient inference that remains valid under alignment uncertainty of permutations. For computational purposes, we develop a linear assignment problem computable in polynomial time and demonstrate that, with high probability, the solution is equivalent to that of the conventional least squares with large computational cost. Extensions to partially permuted designs and ridge regularization are further discussed. Extensive simulations and an application to air-quality data corroborate finite-sample validity, strong power to detect mismatches, and practical scalability.
Paper Structure (52 sections, 30 theorems, 244 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 52 sections, 30 theorems, 244 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and motivation
  3. Problem statement
  4. Overview of contributions
  5. Related works
  6. Shuffled regression
  7. Repro samples method
  8. Organization
  9. Notation
  10. Candidate set of permutations via repro samples
  11. Localization principle
  12. Score-weighted linear assignment problem
  13. Unique recovery of the parameters
  14. Finite-sample statistical inference
  15. Inference on the permutation matrix
  16. ...and 37 more sections

Key Result

Proposition 2.1

In addition to Assumption ass:generic-design, suppose $n-2k \geqslant p$ and $\bm\beta_0\neq \bm 0$ hold. Then, the true permutation $\Pi_0$ uniquely satisfies where $M_{(\Pi \bm{X}, \bm{u}^{\mathrm{rel}})}$ is the orthogonal projector onto the range of $(\Pi \bm{X}, \bm{u}^{\mathrm{rel}})$.

Figures (4)

  • Figure 1: Two-dimensional projections of the union confidence set.
  • Figure 2: Characteristics of the candidate set $\mathcal{C}^{(L)}$.
  • Figure 3: Monte Carlo averages of the coverage probability of the candidate set.
  • Figure 4: Monte Carlo averages of the volume of the confidence sets for the coefficient vector.

Theorems & Definitions (62)

  • Proposition 2.1: Unique recovery in the oracle setting
  • Proposition 2.2
  • Remark 3.1
  • Theorem 4.1: Coverage probability of the candidate set
  • Remark 4.2: Comparison with selective inference
  • Theorem 4.3: Type-I error control for the sparsity test
  • Theorem 4.4: Coverage probability of the confidence set
  • Theorem 4.5: High-probability equivalence
  • Remark 5.1
  • Lemma D.1: Lemma 11 in wang2022finite
  • ...and 52 more