CFTM: Continuous time fractional topic model
Kei Nakagawa, Kohei Hayashi, Yugo Fujimoto
TL;DR
The paper addresses the limitation of static and memoryless dynamic topic models by introducing the Continuous Time Fractional Topic Model (cFTM), which uses fractional Brownian motion with a Hurst index $H$ to drive the evolution of topic and word distributions over continuous time. The approach extends cDTM by incorporating long-term memory or roughness through stochastic differential equations driven by independent fBms, and it shows that parameter estimation remains comparable to LDA when drift terms vanish. Theoretical results assert that cFTM inherits the long-range dependence or roughness of the driving fBm, and empirical evaluation on economic news demonstrates the model’s ability to capture event-driven topic dynamics and memory effects. This work provides a principled framework for modeling time-series topic dynamics with memory, with potential impact on economics, finance, and sociology, while outlining avenues for efficient posterior inference and drift-aware extensions.
Abstract
In this paper, we propose the Continuous Time Fractional Topic Model (cFTM), a new method for dynamic topic modeling. This approach incorporates fractional Brownian motion~(fBm) to effectively identify positive or negative correlations in topic and word distribution over time, revealing long-term dependency or roughness. Our theoretical analysis shows that the cFTM can capture these long-term dependency or roughness in both topic and word distributions, mirroring the main characteristics of fBm. Moreover, we prove that the parameter estimation process for the cFTM is on par with that of LDA, traditional topic models. To demonstrate the cFTM's property, we conduct empirical study using economic news articles. The results from these tests support the model's ability to identify and track long-term dependency or roughness in topics over time.