Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds

TL;DR

Seed-guided topic discovery faces challenges when seeds are out-of-vocabulary and when leveraging powerful pre-trained language models is beneficial. SeeTopic addresses this by coupling a PLM-based general representation with corpus-specific local embeddings, connected through an iterative ensemble ranking that combines both signals to expand category-specific term sets. The method demonstrates improvements in topic coherence, term accuracy, and diversity across SciDocs, Amazon, and Twitter, and shows that incorporating PLMs is particularly advantageous for out-of-vocabulary seeds. The work proposes a PMI-motivated embedding objective and a flexible, mutually exclusive term-enrichment process, offering practical impact for targeted, user-guided topic discovery in heterogeneous domains.

Abstract

Discovering latent topics from text corpora has been studied for decades. Many existing topic models adopt a fully unsupervised setting, and their discovered topics may not cater to users' particular interests due to their inability of leveraging user guidance. Although there exist seed-guided topic discovery approaches that leverage user-provided seeds to discover topic-representative terms, they are less concerned with two factors: (1) the existence of out-of-vocabulary seeds and (2) the power of pre-trained language models (PLMs). In this paper, we generalize the task of seed-guided topic discovery to allow out-of-vocabulary seeds. We propose a novel framework, named SeeTopic, wherein the general knowledge of PLMs and the local semantics learned from the input corpus can mutually benefit each other. Experiments on three real datasets from different domains demonstrate the effectiveness of SeeTopic in terms of topic coherence, accuracy, and diversity.

Figures (2)

• Figure 1: MACC of seed-guided topic discovery methods on in-vocabulary categories and out-of-vocabulary categories.
• Figure 2: Parameter study of SeeTopic measured by topic coherence.