From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions

TL;DR

This work addresses the lack of robust benchmarks for text segmentation in spoken, unstructured content by introducing YTSeg, a large YouTube-based transcription benchmark with chapters, alongside the concept of smart chaptering that combines segmentation with meaningful title generation. It proposes MiniSeg, a lightweight yet competitive hierarchical segmentation model, and extends evaluation to online segmentation and real-time title generation. The paper demonstrates that Wiki-727K pre-training and task adaptation can transfer to YTSeg and even improve performance on related tasks like meeting segmentation, while online settings reveal the importance of controlled future context. The findings have practical implications for structuring video transcripts in real-time applications, enabling clearer navigation and comprehension for users in educational, corporate, and content-creation contexts. Limitations include language scope, single-modality evaluation, and exposure-bias in title generation, suggesting directions toward multi-modal, multilingual smart chaptering in future work.

Abstract

Text segmentation is a fundamental task in natural language processing, where documents are split into contiguous sections. However, prior research in this area has been constrained by limited datasets, which are either small in scale, synthesized, or only contain well-structured documents. In this paper, we address these limitations by introducing a novel benchmark YTSeg focusing on spoken content that is inherently more unstructured and both topically and structurally diverse. As part of this work, we introduce an efficient hierarchical segmentation model MiniSeg, that outperforms state-of-the-art baselines. Lastly, we expand the notion of text segmentation to a more practical "smart chaptering" task that involves the segmentation of unstructured content, the generation of meaningful segment titles, and a potential real-time application of the models.

Figures (5)

• Figure 1: UMAP mcinnes_umap_2018 plot of YTSeg video titles, embedded using Instructor su_one_2023. Category labels are assigned through zero-shot classification with LLaMA 2 touvron_llama_2023.
• Figure 2: The hierarchical architecture of the segmentation model consists of a sentence encoder and a document encoder returning the binary segment boundaries.
• Figure 3: Our offline document encoder is a typical transformer encoder with $N$ transformer layers, each of which applies a full attention mask. Consequently, the encoder can attend to the whole document. In contrast, our online document encoder has $N-M$ layers with causal attention masks that only allow attention to past context, while the initial $M$ layers have attention masks with limited right-side context, that, over these $M$ layers, accumulate to a defined future context size $c$.
• Figure 4: An exemplary output showing duplicate section titles.
• Figure A1: A screenshot of a YouTube video featuring segments as chapters, which form the basis of our new text segmentation benchmark YTSeg.