Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TopicGPT: A Prompt-based Topic Modeling Framework

Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, Mohit Iyyer

TL;DR

TopicGPT introduces a prompt-driven framework that leverages large language models to generate and assign topics from text corpora, producing human-friendly topics with labels, descriptions, and grounded quotes. By combining iterative topic generation with refinement and a self-correcting assignment module, TopicGPT achieves higher alignment with ground-truth topics than traditional baselines like LDA, BERTopic, and SeededLDA across Wiki and Bills datasets. The approach emphasizes interpretability, adaptability, and verifiability, enabling multi-label document-topic associations and reducing misaligned topics through refinement. Robust across prompts and samples, the method offers a practical, human-centered alternative for automated content analysis, while acknowledging current limitations around closed-source model dependence, cost, context limits, and multilinguality.

Abstract

Topic modeling is a well-established technique for exploring text corpora. Conventional topic models (e.g., LDA) represent topics as bags of words that often require "reading the tea leaves" to interpret; additionally, they offer users minimal control over the formatting and specificity of resulting topics. To tackle these issues, we introduce TopicGPT, a prompt-based framework that uses large language models (LLMs) to uncover latent topics in a text collection. TopicGPT produces topics that align better with human categorizations compared to competing methods: it achieves a harmonic mean purity of 0.74 against human-annotated Wikipedia topics compared to 0.64 for the strongest baseline. Its topics are also interpretable, dispensing with ambiguous bags of words in favor of topics with natural language labels and associated free-form descriptions. Moreover, the framework is highly adaptable, allowing users to specify constraints and modify topics without the need for model retraining. By streamlining access to high-quality and interpretable topics, TopicGPT represents a compelling, human-centered approach to topic modeling.

TopicGPT: A Prompt-based Topic Modeling Framework

TL;DR

TopicGPT introduces a prompt-driven framework that leverages large language models to generate and assign topics from text corpora, producing human-friendly topics with labels, descriptions, and grounded quotes. By combining iterative topic generation with refinement and a self-correcting assignment module, TopicGPT achieves higher alignment with ground-truth topics than traditional baselines like LDA, BERTopic, and SeededLDA across Wiki and Bills datasets. The approach emphasizes interpretability, adaptability, and verifiability, enabling multi-label document-topic associations and reducing misaligned topics through refinement. Robust across prompts and samples, the method offers a practical, human-centered alternative for automated content analysis, while acknowledging current limitations around closed-source model dependence, cost, context limits, and multilinguality.

Abstract

Topic modeling is a well-established technique for exploring text corpora. Conventional topic models (e.g., LDA) represent topics as bags of words that often require "reading the tea leaves" to interpret; additionally, they offer users minimal control over the formatting and specificity of resulting topics. To tackle these issues, we introduce TopicGPT, a prompt-based framework that uses large language models (LLMs) to uncover latent topics in a text collection. TopicGPT produces topics that align better with human categorizations compared to competing methods: it achieves a harmonic mean purity of 0.74 against human-annotated Wikipedia topics compared to 0.64 for the strongest baseline. Its topics are also interpretable, dispensing with ambiguous bags of words in favor of topics with natural language labels and associated free-form descriptions. Moreover, the framework is highly adaptable, allowing users to specify constraints and modify topics without the need for model retraining. By streamlining access to high-quality and interpretable topics, TopicGPT represents a compelling, human-centered approach to topic modeling.
Paper Structure (69 sections, 8 equations, 4 figures, 18 tables)

This paper contains 69 sections, 8 equations, 4 figures, 18 tables.

Table of Contents

  1. Introduction
  2. TopicGPT produces higher-quality topics than competing approaches.
  3. TopicGPT produces more interpretable topics.
  4. TopicGPT is customizable to fit user needs:
  5. Open-source LLMs are competent topic assigners but bad topic generators.
  6. Related Work
  7. Topic modeling for content analysis:
  8. LLM-based content analysis:
  9. Comparison to GoalEx:
  10. Methodology
  11. Stage 1: Topic Generation
  12. Generating new topics:
  13. Refining generated topics:
  14. Stage 2: Topic Assignment
  15. Self-correction:
  16. ...and 54 more sections

Figures (4)

  • Figure 1: Overview of TopicGPT. 1) Topic Generation: Given a corpus and some manually-curated example topics, TopicGPT identifies additional topics in each corpus document. The framework then refines the list by merging repeated topics and removing infrequent topics. 2) Topic Assignment: Given the generated topics, TopicGPT assigns the most relevant topic to each document and provides a quote that supports this assignment.
  • Figure 2: Example topic hierarchy for Wiki, with "Architecture & Design" and "Animal Breeds & Husbandry" as the top-level topics generated by TopicGPT. This hierarchical topic structure, in which the upper-level topics are broad enough to encompass more detailed subtopics, allows users to explore topics at different levels of specificity.
  • Figure 3: The number of topics generated over documents processed in the Bills and Wiki corpus. The grey line indicates the number of expected topics, simulated using the empirical distribution of ground-truth topics for the datasets. For both datasets, we see a similar pattern - after a "topic drought" period marked by the dashed red line, the number of initially generated topics (orange line) keeps increasing. However, the final refined topics (blue line) and expected number of topics (grey line) plateau, despite more documents being processed.
  • Figure 4: Mapping interface for human annotators. After reading the instructions in \ref{['tab:instruction']}, the annotators perform mapping between generated and ground-truth topics.