Model-X Change-Point Detection of Conditional Distribution

TL;DR

This paper tackles detecting changes in the conditional distribution $p_{Y|\mathbf{X}}$ across time under covariate drift, a challenge for predictive modeling in high-dimensional nonlinear settings. It introduces MEND, a model-X change-point detection framework that fuses the Conditional Randomization Test (CRT) with a neural network distillation encoder $\phi$ and multiple decoders to model time segments, enabling robust inference and localization of change points. The approach includes EM-based training with optional IRM regularization, a fused-lasso–style procedure to quantify coefficient jumps, and simplified variants for single-change-point scenarios (MEND-mean, MEND-repr). Through extensive simulations and two real-data applications (Beijing air quality and Duke EHRs), MEND demonstrates superior detection power, localization accuracy, and Type-I error control, offering a scalable, distribution-free tool for retraining and policy assessment in dynamic systems.

Abstract

The dynamic nature of many real-world systems can lead to temporal outcome model shifts, causing a deterioration in model accuracy and reliability over time. This requires change-point detection on the outcome models to guide model retraining and adjustments. However, inferring the change point of conditional models is more prone to loss of validity or power than classic detection problems for marginal distributions. This is due to both the temporal covariate shift and the complexity of the outcome model. Also, the existing method of conditional change points detection both have many limitations including linear assumption and low dimension prerequisite which sometimes is not suitable for real world application. To address these challenges, we propose a novel Model-X changE-point detectioN of conditional Distribution (MEND) method computationally enhanced with distillation function for simultaneous change-point detection and localization of the conditional outcome model. We extend and combine our model with neural network to accommodate complex nonlinear and high dimensional situation, which is proved to be valid in both simulation and real data. Theoretical validity of the proposed method is justified. Extensive simulation studies and two real-world examples demonstrate the statistical effectiveness and computational scalability of our method as well as its significant improvements over existing methods.

Figures (4)

• Figure 1: Comparison of Type-$\text{II}$ error rate and average p-value rate across different methods(linear)
• Figure 2: Comparison of Type-$\text{II}$ error rate and average p-value rate across different methods(non-linear)
• Figure 3: Application in Beijing Air Quality Data
• Figure 4: Comparison of Type-I error rate, power and accuracy rate across different scenarios and methods (linear)

Theorems & Definitions (3)

• Theorem 1: Strict validity of CRT
• Proposition 1
• proof : Proof of Proposition \ref{['prop:crt']}