Detecting Abrupt Changes in Point Processes: Fundamental Limits and Applications

TL;DR

This work develops a theory and practical algorithms for detecting abrupt, transient changes in the rate $\Lambda(t)$ of an unknown, possibly random, point process. By forecasting a counterfactual trajectory with local polynomial approximations and applying derivative-thresholding to the observed counts, the authors derive sharp information-theoretic limits: changes with magnitude $A$ exceeding the natural smoothness scale (roughly $\sqrt{B}$) are detectable with vanishing delay, while smaller changes are statistically impossible to detect. The framework extends to complex networks via the SI epidemic model, establishing a phase transition at $\alpha=1/2$ for detecting high-degree vertices and, with multiple cascades, exact estimation of those vertices. Empirical results on synthetic data, as well as real COVID-19 case data, illustrate the method’s robustness and practical utility for identifying super-spreading events and abrupt shifts in event rates. Overall, the paper provides a scalable, online detection mechanism with solid theoretical guarantees for non-stationary point processes across diverse domains.

Abstract

We consider the problem of detecting abrupt changes (i.e., large jump discontinuities) in the rate function of a point process. The rate function is assumed to be fully unknown, non-stationary, and may itself be a random process that depends on the history of event times. We show that abrupt changes can be accurately identified from observations of the point process, provided the changes are sharper than the "smoothness'' of the rate function before the abrupt change. This condition is also shown to be necessary from an information-theoretic point of view. We then apply our theory to several special cases of interest, including the detection of significant changes in piecewise smooth rate functions and detecting super-spreading events in epidemic models on graphs. Finally, we confirm the effectiveness of our methods through a detailed empirical analysis of both synthetic and real datasets.

Figures (8)

• Figure 1: Phase diagram for the possibility and impossibility of estimating high-degree vertices in a graph $G \in {\mathcal{G}}$. Dark blue region: Achievability region for the algorithm of prior work (second derivative thresholding) mossel2024finding, for which high-degree vertices can be estimated from more than $1/(\alpha - 3/4)$ cascade traces. Light blue region: High-degree vertices can be estimated from more than $1/(2\alpha - 1)$ cascade traces (proven in Theorem \ref{['thm:estimating_high_degrees']}). Red region: Estimating high-degree vertices is impossible, even when $G$ is known to be a tree mossel2024finding. Green region: Full recovery of $G$ is possible if $G$ is a tree, hence estimation of high-degree vertices is also possible mossel2024findingACFKP13_trace_complexity. White region: Regimes with small gaps between the bounds provided by our analysis.
• Figure 2: Discrete derivatives of $N(t)$ with $\delta = 0.2$. The red dotted line represents the jump occurring at $t_0 = 9$.
• Figure 3: Heatmaps of the difference between the estimated and true value of $t_0$, for $A \in \{20000, 40000, 60000, 80000 \}$. In the subfigure captions above, $k_{\min}$ and $\delta_{\min}$ denote the values of $k, \delta$ that minimize the estimation error in the heatmap range.
• Figure 4: Discrete derivatives of $N(t)$ with $\delta = 0.4$, for a SI process spreading on a balanced binary tree on approximately 500,000 vertices with a single vertex of degree $3002$. The red dotted line represents the infection time of this high-degree vertex, which is at $t = 23.00$.
• Figure 5: Heatmaps of the difference between estimated and true values of the infection time of the (unique) high-degree vertex, for $D \in \{ 2000, 4000, 6000, 8000 \}$, averaged over 200 independent simulations. In the subfigure captions above, $k_{\min}$ and $\delta_{\min}$ denote the values of $k, \delta$ that minimize the estimation error in the heatmap range.

Theorems & Definitions (47)

• Proposition 2.1
• Theorem 2.2: Informal and simplified version of Theorem \ref{['thm:derivative_thresholding']}
• Remark 1: Asymptotic notation
• Remark 2: Non-asymptotic results
• Remark 3: Applicability to more complex models
• Definition 2.1: Smooth + jump rate function
• Definition 2.3
• Theorem 2.3
• Definition 2.4
• Theorem 2.4: Detecting high-degree vertices