Topic Segmentation Using Generative Language Models

TL;DR

This work investigates topic segmentation using generative language models by introducing an overlapping and recursive prompting strategy that outputs unedited boundary indices. It evaluates LLM-based segmentation against embedding-based baselines across four datasets, showing that LLMs excel at nuanced segment boundaries but may struggle with very noisy or very clear-cut cases where traditional methods perform reliably. A boundary-similarity metric is adopted to enable robust comparison, and a token-efficient prompting framework is proposed to handle long documents within context limits. The findings suggest meaningful potential for LLM-driven segmentation in long documents and summaries, while also highlighting the need for larger, diverse annotated datasets and improved prompt engineering to ensure reliability and reproducibility.

Abstract

Topic segmentation using generative Large Language Models (LLMs) remains relatively unexplored. Previous methods use semantic similarity between sentences, but such models lack the long range dependencies and vast knowledge found in LLMs. In this work, we propose an overlapping and recursive prompting strategy using sentence enumeration. We also support the adoption of the boundary similarity evaluation metric. Results show that LLMs can be more effective segmenters than existing methods, but issues remain to be solved before they can be relied upon for topic segmentation.

Figures (1)

• Figure 1: The overlapping and recursive prompting strategy for segmentation. In (a), a long document is split into overlapping sections with sentence boundaries enumerated. In (b), each section is segmented by the LLM. In (c), the segments are joined. Finally, in (d), each segment is validated to ensure it is not too long or too short.