A Bayesian approach to modeling topic-metadata relationships
P. Schulze, S. Wiegrebe, P. W. Thurner, C. Heumann, M. Aßenmacher
TL;DR
This paper tackles estimating relationships between latent topics and metadata in the Structural Topic Model by showing that the standard method of composition, which regresses sampled topic proportions on covariates via linear regression, can yield implausible predictions and mixes Bayesian with frequentist inference. It introduces two advances: (i) Beta regression to respect the $0<\theta_{d,k}<1$ bounds of topic proportions and (ii) a fully Bayesian Beta regression framework to obtain coherent posterior predictive distributions for topic proportions across covariate levels. The approach is demonstrated on Twitter data from German MPs (Sept 2017–Apr 2020), revealing more interpretable, uncertainty-aware relationships between topics and covariates such as unemployment and immigrant share, and providing richer quantification of between-MP variability through posterior predictive checks. The methods extend beyond STM to other topic-model frameworks, improving reliability of topic–metadata analyses in social science contexts.
Abstract
The objective of advanced topic modeling is not only to explore latent topical structures, but also to estimate relationships between the discovered topics and theoretically relevant metadata. Methods used to estimate such relationships must take into account that the topical structure is not directly observed, but instead being estimated itself in an unsupervised fashion, usually by common topic models. A frequently used procedure to achieve this is the method of composition, a Monte Carlo sampling technique performing multiple repeated linear regressions of sampled topic proportions on metadata covariates. In this paper, we propose two modifications of this approach: First, we substantially refine the existing implementation of the method of composition from the R package stm by replacing linear regression with the more appropriate Beta regression. Second, we provide a fundamental enhancement of the entire estimation framework by substituting the current blending of frequentist and Bayesian methods with a fully Bayesian approach. This allows for a more appropriate quantification of uncertainty. We illustrate our improved methodology by investigating relationships between Twitter posts by German parliamentarians and different metadata covariates related to their electoral districts, using the Structural Topic Model to estimate topic proportions.