A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics

TL;DR

This work addresses non-stationary count time-series by introducing Non-Stationary Poisson-Gamma Dynamical Systems (NS-PGDS), which extends PGDS with time-varying transition dynamics modeled by Dirichlet Markov chains. It develops three flexible constructions—Dir-Dir, Dir-Gam-Dir, and PR-Gam-Dir—to capture smooth evolution, mutation among latent factors, and sparse transition patterns, respectively, and provides a fully-conjugate Gibbs sampler via Dirichlet-Multinomial-Beta data augmentation. Empirical results on ICEWS, NIPS, USEI, and COVID datasets show improved predictive performance and reveal interpretable, time-adaptive latent structures. The approach offers a principled, scalable framework for modeling evolving dependencies in noisy count data with practical implications for forecasting and exploratory analysis of complex temporal processes.

Abstract

Probabilistic approaches for handling count-valued time sequences have attracted amounts of research attentions because their ability to infer explainable latent structures and to estimate uncertainties, and thus are especially suitable for dealing with \emph{noisy} and \emph{incomplete} count data. Among these models, Poisson-Gamma Dynamical Systems (PGDSs) are proven to be effective in capturing the evolving dynamics underlying observed count sequences. However, the state-of-the-art PGDS still fails to capture the \emph{time-varying} transition dynamics that are commonly observed in real-world count time sequences. To mitigate this gap, a non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time, and the evolving transition matrices are modeled by sophisticatedly-designed Dirichlet Markov chains. Leveraging Dirichlet-Multinomial-Beta data augmentation techniques, a fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance due to its capacity to learn non-stationary dependency structure captured by the time-evolving transition matrices.

Figures (6)

• Figure 1: An example illustrates the Poisson-gamma dynamical systems with non-stationary transition kernels. The three gamma dynamic processes independently evolve over time during the ($i-1$)-th interval. During $i$-th interval, $\theta_1^{(t)}$ and $\theta_2^{(t)}$ gradually starts to interact with each other while $\theta_3^{(t)}$ remains independent to the other two dimensions. During ($i+1$)-th interval all the three latent components start to interact with each other.
• Figure 2: Graphical representation of the NS-PGDS. The time interval is divided into equally-spaced sub-intervals. Each sub-interval contains $M$ time steps. The transition dynamics is stationary within a sub-interval. In particular, the transition matrices evolve over sub-intervals via Dirichlet Markov processes while latent factors evolve over time steps via Eq.(\ref{['ns-pgds']}).
• Figure 3: Diagrams of the proposed Dirichlet Markov constructions. (a) is the Dir-Dir construction. (b) is the Dir-Gam-Dir construction which takes mutation into account. (c) illustrates the PR-Gam-Dir construction which adopts Poisson randomized gamma distribution and can be equivalently represented as Eq.(\ref{['pr_gam_1']}) and Eq.(\ref{['pr_gam_2']}).
• Figure 4: Latent factors inferred by NS-PGDS. (a) and (b) illustrate the top 2 latent factors inferred from ICEWS dataset, (a) corresponds to Iraq war and (b) corresponds to the Six-Party Talks. (c) illustrates the evolving trends of the top 5 latent factors inferred from NIPS dataset.
• Figure 5: Transition matrices inferred from NIPS dataset. (a) illustrates the transition matrix inferred by the PGDS. (b)-(f) illustrate the time-varying transition matrices inferred by the NS-PGDS.

Theorems & Definitions (2)

• Lemma 1
• Lemma 2