Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation

Steffen Freisinger, Philipp Seeberger, Thomas Ranzenberger, Tobias Bocklet, Korbinian Riedhammer

TL;DR

This paper tackles the absence of structure in long speech transcripts by introducing a hierarchical ToC generation method that yields multi-level topic boundaries. It employs LLM-driven ToC generation, comparing zero-shot prompting with LoRA-based fine-tuning, and optionally infusing inter-sentence pauses as acoustic cues. The authors adapt a hierarchical evaluation metric to jointly assess coarse-to-fine segmentations and demonstrate improvements over strong baselines on English and multilingual datasets. The work advances accessible transcript navigation and downstream applications such as IR and RAG by providing structured, multi-level transcripts.

Abstract

Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hierarchical topic segmentation in transcripts, generating multi-level tables of contents that capture both topic and subtopic boundaries. We compare zero-shot prompting and LoRA fine-tuning on large language models, while also exploring the integration of high-level speech pause features. Evaluations on English meeting recordings and multilingual lecture transcripts (Portuguese, German) show significant improvements over established topic segmentation baselines. Additionally, we adapt a common evaluation measure for multi-level segmentation, taking into account all hierarchical levels within one metric.

Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation

TL;DR

This paper tackles the absence of structure in long speech transcripts by introducing a hierarchical ToC generation method that yields multi-level topic boundaries. It employs LLM-driven ToC generation, comparing zero-shot prompting with LoRA-based fine-tuning, and optionally infusing inter-sentence pauses as acoustic cues. The authors adapt a hierarchical evaluation metric to jointly assess coarse-to-fine segmentations and demonstrate improvements over strong baselines on English and multilingual datasets. The work advances accessible transcript navigation and downstream applications such as IR and RAG by providing structured, multi-level transcripts.

Abstract

Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hierarchical topic segmentation in transcripts, generating multi-level tables of contents that capture both topic and subtopic boundaries. We compare zero-shot prompting and LoRA fine-tuning on large language models, while also exploring the integration of high-level speech pause features. Evaluations on English meeting recordings and multilingual lecture transcripts (Portuguese, German) show significant improvements over established topic segmentation baselines. Additionally, we adapt a common evaluation measure for multi-level segmentation, taking into account all hierarchical levels within one metric.
Paper Structure (17 sections, 3 equations, 2 figures, 3 tables)

This paper contains 17 sections, 3 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Spoken Document Segmentation
  4. Hierarchical Topic Segmentation
  5. Table-of-Contents Generation
  6. Method
  7. Problem Definition
  8. ToC-LLM: Table-of-Contents Generation
  9. Infusing Acoustic Cues
  10. Modeling and Implementation
  11. Experiments
  12. Datasets
  13. Hierarchical Evaluation
  14. Baselines
  15. Linear Segmentation Results
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of our approach with an example from Videoaula (translated): First, we generate transcripts using an ASR system and apply forced alignment (FA) to synchronize the transcripts with the audio. Then, we extract inter-sentence pause durations and prompt the LLM along with the text. The output is a table of contents that shows the topic boundaries.
  • Figure 2: Level-specific $B$ scores for each reference (rows) to hypothesis (column) level combination. Average over all samples of ToC-Nemo + Pause on Videoaula