Large Language Model and Formal Concept Analysis: a comparative study for Topic Modeling

TL;DR

The paper experimentally compares a formal concept analysis approach (CREA) for topic modeling with a prompt-driven large language model (GPT-5) across two domain datasets. It details CREA's semantic preprocessing, structural analysis, and clustering steps, and contrasts them with GPT-5's three-prompt zero-shot pipeline for topic generation, merging, and labeling. Results show that CREA often yields imbalanced clusters and requires extensive parameter tuning, while GPT-5 tends to produce balanced, coherent topics with interpretable labels but raises reproducibility and transparency concerns due to its black-box nature. The work highlights trade-offs between traceability and ease of use, suggesting method choice should be guided by the need for interpretability, scale, and application context in topic modeling tasks.

Abstract

Topic modeling is a research field finding increasing applications: historically from document retrieving, to sentiment analysis and text summarization. Large Language Models (LLM) are currently a major trend in text processing, but few works study their usefulness for this task. Formal Concept Analysis (FCA) has recently been presented as a candidate for topic modeling, but no real applied case study has been conducted. In this work, we compare LLM and FCA to better understand their strengths and weakneses in the topic modeling field. FCA is evaluated through the CREA pipeline used in past experiments on topic modeling and visualization, whereas GPT-5 is used for the LLM. A strategy based on three prompts is applied with GPT-5 in a zero-shot setup: topic generation from document batches, merging of batch results into final topics, and topic labeling. A first experiment reuses the teaching materials previously used to evaluate CREA, while a second experiment analyzes 40 research articles in information systems to compare the extracted topics with the underling subfields.

Figures (8)

• Figure 1: The steps of the CREA pipeline.
• Figure 2: The substeps of the FCA step (PII.1) in the CREA pipeline.
• Figure 3: Silhouette score on clusters from Abstracts concerning Medium vs. High Strategy ($\beta = 1.00$)
• Figure 4: Davies-Bouldin index on clusters from Abstracts concerning Medium vs. High Strategy ($\beta = 1.00$)
• Figure 5: Calinski-Harabasz index on clusters from Abstracts concerning Medium vs. High Strategy ($\beta = 1.00$)