Sequential Change Detection Under A Markov Setup With Unknown Pre-Change and Post-Change Distributions

TL;DR

This work tackles sequential change detection when both pre-change and post-change distributions are unknown in a Markov setting. It extends Page’s CUSUM by leveraging an empirical pre-change distribution and a strongly pointwise universal code to estimate the post-change distribution, within stationary ergodic finite-order Markov sources satisfying Berk's mixing. The authors derive finite-sample and asymptotic performance guarantees, including an upper bound on false alarms, guaranteed termination under the post-change distribution, and an asymptotically optimal detection delay under Lorden’s criterion (via $rac{|\,\log \alpha\,|}{D(\nmu_1 \\| \nmu_0^ ext{hat}) - \lambda}$). The results unify and extend prior work (i.e., Malik–Bansal, Jacob–Bansal) from i.i.d. and memoryless models to Markov sources, providing a robust framework for sequential detection with unknown distributions and offering avenues for further generalization to Lai/Pollak-type criteria and broader dependency structures.

Abstract

In this work we extend the results developed in 2022 for a sequential change detection algorithm making use of Page's CUSUM statistic, the empirical distribution as an estimate of the pre-change distribution, and a universal code as a tool for estimating the post-change distribution, from the i.i.d. case to the Markov setup.