High-Dimensional Differential Parameter Inference in Exponential Family using Time Score Matching

TL;DR

This work tackles differential inference in time-varying exponential-family models by directly estimating the time derivative of the natural parameters, $\partial_t \boldsymbol{\theta}^*(t)$, rather than the full trajectory $\boldsymbol{\theta}^*(t)$. It introduces time score matching, which represents the time score $\partial_t \log q_t(\mathbf{x})$ as a linear function of the differential parameter, enabling direct, scalable estimation in high dimensions. The authors establish consistency of the regularized score-matching objective (SparTSM) and derive finite-sample Gaussian approximations for a debiased variant (SparTSM+), providing asymptotically valid inference under sparsity and restricted eigenvalue conditions. The methods are validated on simulations and real data (109th US Senate), showing effective recovery of differential structure and reliable confidence intervals, with competitive performance against existing time-varying approaches. Overall, the approach enables efficient, interpretable discovery of time-local changes in high-dimensional probabilistic models without exhaustively fitting time-varying parameters.

Abstract

This paper addresses differential inference in time-varying parametric probabilistic models, like graphical models with changing structures. Instead of estimating a high-dimensional model at each time point and estimating changes later, we directly learn the differential parameter, i.e., the time derivative of the parameter. The main idea is treating the time score function of an exponential family model as a linear model of the differential parameter for direct estimation. We use time score matching to estimate parameter derivatives. We prove the consistency of a regularized score matching objective and demonstrate the finite-sample normality of a debiased estimator in high-dimensional settings. Our methodology effectively infers differential structures in high-dimensional graphical models, verified on simulated and real-world datasets. The code reproducing our experiments can be found at: https://github.com/Leyangw/tsm.

Figures (5)

• Figure 1: SparTSM compared with Loggle. Estimating the differential parameter vs. Estimating the time-varying parameter. The overall graphical model ${\boldsymbol{\Theta}}$ is non-sparse.
• Figure 2: ROC curves of SparTSM, Density Ratio, and Loggle.
• Figure 3: Gaussian approximation of SparTSM+. The left shows a bar plot and a QQ-plot is on the right.
• Figure 4: Testing Power plot. $\mathcal{H}_0 : \partial_t\Theta_{1,2}(t) = 0$.
• Figure 5: The differential graph estimated from 109th senate voting dataset. Red edge indicate a positive $\alpha_{i,j} \approx \partial_t \Theta^*_{i,j}(t)$. The widths of edges are proportional to $|\alpha_{i,j}|$. Democrats, republicans, independents are colored in blue, red and green respectively. Party whips are marked by a black rectangle.

Theorems & Definitions (31)

• Proposition 3.1
• Theorem 4.1
• Theorem 6.4
• Theorem 6.5: GAB
• Corollary 6.7
• Theorem D.1
• Lemma D.2
• proof
• Lemma D.3
• proof