Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Covariance scanning for adaptively optimal change point detection in high-dimensional linear models

Haeran Cho, Housen Li

TL;DR

This work studies the AMOC change-point problem in high-dimensional linear models with Gaussian regressors, uncovering a phase transition driven by the inherent sparsity $\mathfrak{s}=|\bm{\Sigma}^{1/2}\bm{\delta}|_0$. It introduces two covariance-scanning detectors, McScan (sparse regime) and QcScan (dense regime), and an adaptive amalgam OcScan that achieves minimax-optimal performance uniformly over sparsity levels with linear-time computation $O(n p)$. The authors also provide post-detection estimation and a refinement step (OcScan.R) that improve localisation when $\bm\delta$ is sparse, and extend guarantees to rank-deficient $\bm\Sigma$. Extensive simulations corroborate the theory, highlighting computational efficiency and stable performance across regimes and covariance structures.

Abstract

This paper investigates the detection and estimation of a single change in high-dimensional linear models. We derive minimax lower bounds for the detection boundary and the estimation rate, which uncover a phase transition governed by the sparsity of the covariance-weighted differential parameter. This form of "inherent sparsity" captures a delicate interplay between the covariance structure of the regressors and the change in regression coefficients on the detectability of a change point. Complementing the lower bounds, we introduce two covariance scanning-based methods, McScan and QcSan, which achieve minimax optimal performance (up to possible logarithmic factors) in the sparse and the dense regimes, respectively. In particular, QcScan is the first method shown to achieve consistency in the dense regime and further, we devise a combined procedure which is adaptively minimax optimal across sparse and dense regimes without the knowledge of the sparsity. Computationally, covariance scanning-based methods avoid costly computation of Lasso-type estimators and attain worst-case computation complexity that is linear in the dimension and sample size. Additionally, we consider the post-detection estimation of the differential parameter and the refinement of the change point estimator. Simulation studies support the theoretical findings and demonstrate the computational and statistical efficiency of the proposed covariance scanning methods.

Covariance scanning for adaptively optimal change point detection in high-dimensional linear models

TL;DR

This work studies the AMOC change-point problem in high-dimensional linear models with Gaussian regressors, uncovering a phase transition driven by the inherent sparsity . It introduces two covariance-scanning detectors, McScan (sparse regime) and QcScan (dense regime), and an adaptive amalgam OcScan that achieves minimax-optimal performance uniformly over sparsity levels with linear-time computation . The authors also provide post-detection estimation and a refinement step (OcScan.R) that improve localisation when is sparse, and extend guarantees to rank-deficient . Extensive simulations corroborate the theory, highlighting computational efficiency and stable performance across regimes and covariance structures.

Abstract

This paper investigates the detection and estimation of a single change in high-dimensional linear models. We derive minimax lower bounds for the detection boundary and the estimation rate, which uncover a phase transition governed by the sparsity of the covariance-weighted differential parameter. This form of "inherent sparsity" captures a delicate interplay between the covariance structure of the regressors and the change in regression coefficients on the detectability of a change point. Complementing the lower bounds, we introduce two covariance scanning-based methods, McScan and QcSan, which achieve minimax optimal performance (up to possible logarithmic factors) in the sparse and the dense regimes, respectively. In particular, QcScan is the first method shown to achieve consistency in the dense regime and further, we devise a combined procedure which is adaptively minimax optimal across sparse and dense regimes without the knowledge of the sparsity. Computationally, covariance scanning-based methods avoid costly computation of Lasso-type estimators and attain worst-case computation complexity that is linear in the dimension and sample size. Additionally, we consider the post-detection estimation of the differential parameter and the refinement of the change point estimator. Simulation studies support the theoretical findings and demonstrate the computational and statistical efficiency of the proposed covariance scanning methods.

Paper Structure

This paper contains 59 sections, 24 theorems, 194 equations, 24 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notations.
  3. Minimax lower bound
  4. Isotropic case
  5. General covariance
  6. Upper bounds
  7. Sparse regime
  8. Dense regime
  9. Post-detection estimation of the differential parameter
  10. When the differential parameter is sparse
  11. For the differential parameter of arbitrary sparsity
  12. Adaption to unknown sparsity, refinement and computational complexity
  13. Adaptively optimal change point detection
  14. Refined change point estimation
  15. Runtime and memory
  16. ...and 44 more sections

Key Result

Theorem 1

Let $\sigma > 0$ and $n$ be sufficiently large. We consider the case where $\bm\Sigma = \mathbf I$. If for some $\nu \in (0, 1/2)$, then it holds, for every $\tau \in (0, 1/6]$, where the infimum is taken over all measurable functions $\psi:\mathbb{R}^{n \times (1+p)}\to[0,1]$.

Figures (24)

  • Figure 1: Estimation performance of MOSEG cho2022high, CHARCOAL (gao2022sparse; only applicable when $n > p$), and the proposed methods: McScan (Section \ref{['sec:sparse']}; a variant of cho2024detection), QcScan (Section \ref{['sec:dense']}), OcScan (Section \ref{['sec:adaptive']}) and OcScan.R (Section \ref{['sec:refine']}), in the isotropic setting ($\bm\Sigma = \mathbf I$) for varying $\mathfrak{s}$ ($x$-axis) and sample size $n$ (left to right) while $p = 900$ and $\vert \bm\delta \vert_2 = 8$. Results are based on 1000 repetitions, with median error curves shown alongside shaded regions representing the interquartile range. The vertical dashed lines correspond to where $\mathfrak{s} = \sqrt{p{\color{black}\log\log(n)}}$. The $x$-axis is displayed on a log scale, and the $y$-axis on a squared root scale. This is excerpted from Figure \ref{['f:iderr']} in \ref{['sec:sim']}.
  • Figure 2: Estimation performance of MOSEG cho2022high, CHARCOAL gao2022sparse, and the proposed methods: McScan (Section \ref{['sec:sparse']}; a variant of cho2024detection), QcScan (Section \ref{['sec:dense']}), OcScan (Section \ref{['sec:adaptive']}) and OcScan.R (Section \ref{['sec:refine']}), when $\bm\Sigma = [0.6^{\vert i - j \vert}]_{i, j}$. Taken from (M2) in \ref{['sec:sim']}, we set $n = 300$, $p = 200$, $\bm\beta_0 + \bm\beta_1 = \mathbf 0$ and $\vert\bm\Sigma^{1/2}\bm\delta\vert_2 = 2$. In the left, the $x$-axis denotes the standard sparsity $\mathfrak{s}_\delta = \vert \bm\delta \vert_0$ while in the middle, it is the inherent sparsity $\mathfrak{s} = \vert \bm\Sigma^{1/2}\bm\delta \vert_0$. In each scenario, results are based on 1000 repetitions, with median error curves shown alongside shaded regions representing the interquartile range. The right panel plots the quantity $\min( \vert\bm\Sigma^{1/2}\bm\delta\vert_0, \sqrt{p{\color{black}\log\log (n)}} )$ as either $\mathfrak{s}_\delta$ or $\mathfrak{s}$ varies. This quantity is the influencing factor, save logarithmic factors, appearing in the minimax rate of estimation (see \ref{['tab:minimax']}), and represents a proxy of the difficulty of the estimation problem. The vertical dashed lines mark where $\mathfrak{s}_\delta = \sqrt{p {\color{black}\log\log (n)}}$ or $\mathfrak{s} = \sqrt{p{\color{black}\log\log (n)}}$. The $x$-axis is shown on a log scale, and the $y$-axis on a squared root scale.
  • Figure 3: Estimation performance of MOSEG, CHARCOAL, McScan, QcScan, OcScan and OcScan.R in (M1); CHARCOAL is only applicable in the last column where $n = 1200 > p = 900$. The $x$-axis denotes the inherent sparsity $\mathfrak{s}$ which coincides with $\mathfrak{s}_\delta$. In each scenario, the results are based on 1000 repetitions, with median error curves shown alongside shaded regions representing the interquartile range. The vertical dashed lines mark where $\mathfrak{s} = \sqrt{p{\color{black}\log\log(n)}}$. The $x$-axis is shown on a log scale, and the $y$-axis on a squared root scale.
  • Figure 4: Computation time of MOSEG, CHARCOAL, McScan, QcScan, OcScan and OcScan.R in (M1); CHARCOAL is only applicable in the last column where $n = 1200 > p = 900$. The curves plot the median runtimes (on 1.0 GHz processors) over 1000 repetitions when $\rho = 2$. The runtimes for $\rho \in\{1,4\}$ are very similar. Both axes are on a log scale.
  • Figure 5: Estimation performance of MOSEG, CHARCOAL, McScan, QcScan, OcScan and OcScan.R in modified (M1) where $n = 1200$, $p = 900$ and $\bm\Sigma = \mathbf U_r \mathbf U_r^\top$ with varying $r \in \{p/4, p/2, p\}$, where $\mathbf U_r \in \mathbb{R}^{p \times r}$ is randomly generated with orthonormal columns. The $x$-axis denotes $\mathfrak{s}_\delta = \vert \bm\delta \vert_0$. In each scenario, the results are based on 1000 repetitions, with median error curves shown alongside shaded regions representing the interquartile range. The vertical dashed lines mark where $\mathfrak{s}_\delta = \sqrt{p{\color{black}\log\log(n)}}$. The $x$-axis is shown on a log scale, and the $y$-axis on a squared root scale.
  • ...and 19 more figures

Theorems & Definitions (39)

  • Theorem 1
  • Lemma 2: Lemma 1 of cho2024detection, cho2024detection
  • Remark 2.1
  • Theorem 3
  • Remark 3.1
  • Proposition 4
  • Theorem 5
  • Remark 3.2
  • Proposition 6
  • Proposition 7
  • ...and 29 more