SciTopic: Enhancing Topic Discovery in Scientific Literature through Advanced LLM

TL;DR

This work proposes an advanced topic discovery method enhanced by LLMs to improve scientific topic identification, namely SciTopic, which outperforms the state-of-the-art (SOTA) scientific topic discovery methods, enabling researchers to gain deeper and faster insights.

Abstract

Topic discovery in scientific literature provides valuable insights for researchers to identify emerging trends and explore new avenues for investigation, facilitating easier scientific information retrieval. Many machine learning methods, particularly deep embedding techniques, have been applied to discover research topics. However, most existing topic discovery methods rely on word embedding to capture the semantics and lack a comprehensive understanding of scientific publications, struggling with complex, high-dimensional text relationships. Inspired by the exceptional comprehension of textual information by large language models (LLMs), we propose an advanced topic discovery method enhanced by LLMs to improve scientific topic identification, namely SciTopic. Specifically, we first build a textual encoder to capture the content from scientific publications, including metadata, title, and abstract. Next, we construct a space optimization module that integrates entropy-based sampling and triplet tasks guided by LLMs, enhancing the focus on thematic relevance and contextual intricacies between ambiguous instances. Then, we propose to fine-tune the textual encoder based on the guidance from the LLMs by optimizing the contrastive loss of the triplets, forcing the text encoder to better discriminate instances of different topics. Finally, extensive experiments conducted on three real-world datasets of scientific publications demonstrate that SciTopic outperforms the state-of-the-art (SOTA) scientific topic discovery methods, enabling researchers to gain deeper and faster insights.

Figures (7)

• Figure 1: Overview of the proposed SciTopic framework. The framework comprises three key stages: a) Textual encoder, where title, abstract, and metadata are separately encoded and concatenated to form a comprehensive document representation; b) LLM-guided clustering, which leverages LLM-guided triplet tasks and entropy-based sampling to handle thematically ambiguous documents, enhancing clustering precision through LLM feedback; and c) Fine-tuning, where the embedding model is optimized using LLM triplet feedback to produce final clustering results with improved thematic relevance and coherence.
• Figure 2: Illustration of class-based TF-IDF analysis.
• Figure 3: WordCloud visualization on AI-DM of top 50 words per topic across LDA, BERTopic, Fastopic, and SciTopic, when topic number equals 10.
• Figure 4: Temporal evolution of all topics on AI-DM.
• Figure 5: Parameter sensitivity analysis on Topic Coherence and Topic Diversity.