Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seeded Poisson Factorization: leveraging domain knowledge to fit topic models

Bernd Prostmaier, Jan Vávra, Bettina Grün, Paul Hofmarcher

TL;DR

S seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization (PF) framework by incorporating domain knowledge through seed words, achieves superior performance compared to alternative guided probabilistic topic models in terms of computational efficiency and classification performance.

Abstract

Topic models are widely used for discovering latent thematic structures in large text corpora, yet traditional unsupervised methods often struggle to align with pre-defined conceptual domains. This paper introduces seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization (PF) framework by incorporating domain knowledge through seed words. SPF enables a structured topic discovery by modifying the prior distribution of topic-specific term intensities, assigning higher initial rates to pre-defined seed words. The model is estimated using variational inference with stochastic gradient optimization, ensuring scalability to large datasets. We present in detail the results of applying SPF to an Amazon customer feedback dataset, leveraging pre-defined product categories as guiding structures. SPF achieves superior performance compared to alternative guided probabilistic topic models in terms of computational efficiency and classification performance. Robustness checks highlight SPF's ability to adaptively balance domain knowledge and data-driven topic discovery, even in case of imperfect seed word selection. Further applications of SPF to four additional benchmark datasets, where the corpus varies in size and the number of topics differs, demonstrate its general superior classification performance compared to the unseeded PF model.

Seeded Poisson Factorization: leveraging domain knowledge to fit topic models

TL;DR

S seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization (PF) framework by incorporating domain knowledge through seed words, achieves superior performance compared to alternative guided probabilistic topic models in terms of computational efficiency and classification performance.

Abstract

Topic models are widely used for discovering latent thematic structures in large text corpora, yet traditional unsupervised methods often struggle to align with pre-defined conceptual domains. This paper introduces seeded Poisson Factorization (SPF), a novel approach that extends the Poisson Factorization (PF) framework by incorporating domain knowledge through seed words. SPF enables a structured topic discovery by modifying the prior distribution of topic-specific term intensities, assigning higher initial rates to pre-defined seed words. The model is estimated using variational inference with stochastic gradient optimization, ensuring scalability to large datasets. We present in detail the results of applying SPF to an Amazon customer feedback dataset, leveraging pre-defined product categories as guiding structures. SPF achieves superior performance compared to alternative guided probabilistic topic models in terms of computational efficiency and classification performance. Robustness checks highlight SPF's ability to adaptively balance domain knowledge and data-driven topic discovery, even in case of imperfect seed word selection. Further applications of SPF to four additional benchmark datasets, where the corpus varies in size and the number of topics differs, demonstrate its general superior classification performance compared to the unseeded PF model.

Paper Structure

This paper contains 18 sections, 10 equations, 3 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The seeded Poisson factorization model
  3. Inference
  4. Variational inference
  5. Post-processing final inference
  6. Computational details
  7. Empirical results
  8. Application to the Amazon Reviews dataset
  9. Topic assessment
  10. Classification performance
  11. Comparison to existing methods and scalability
  12. Robustness checks
  13. Effect of the number of seed words.
  14. Effect of learning rate, epochs and batch size.
  15. Effect of varying the number of topics $K$.
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Architecture of the seeded Poisson Factorization (SPF) topic model.
  • Figure 2: Directed graphical representation of the SPF model. Shaded nodes are observed, transparent nodes are latent variables, double circles indicate deterministic transformations of parent nodes and points are fixed parameters.
  • Figure 3: Processing time for the bootstrap experiment.