Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization

Frederic Kirstein, Terry Ruas, Robert Kratel, Bela Gipp

TL;DR

The work addresses salience identification and personalization in meeting summaries by proposing a three-stage RAG-based pipeline that enriches transcripts with information inferred from supplementary materials. It also adds a personalization protocol that extracts participant personas to tailor summaries for specific readers. Through the MS-AMI dataset and extensive experiments using GPT-4 Turbo and smaller backbones, the approach demonstrates improved informativeness, relevance, and reduced hallucinations for general and personalized summaries, while exploring practical deployment trade-offs. The findings highlight the value of distributing multi-source challenges across dedicated modules and pave the way for on-device and cost-aware implementations in real-world settings.

Abstract

Meeting summarization is crucial in digital communication, but existing solutions struggle with salience identification to generate personalized, workable summaries, and context understanding to fully comprehend the meetings' content. Previous attempts to address these issues by considering related supplementary resources (e.g., presentation slides) alongside transcripts are hindered by models' limited context sizes and handling the additional complexities of the multi-source tasks, such as identifying relevant information in additional files and seamlessly aligning it with the meeting content. This work explores multi-source meeting summarization considering supplementary materials through a three-stage large language model approach: identifying transcript passages needing additional context, inferring relevant details from supplementary materials and inserting them into the transcript, and generating a summary from this enriched transcript. Our multi-source approach enhances model understanding, increasing summary relevance by ~9% and producing more content-rich outputs. We introduce a personalization protocol that extracts participant characteristics and tailors summaries accordingly, improving informativeness by ~10%. This work further provides insights on performance-cost trade-offs across four leading model families, including edge-device capable options. Our approach can be extended to similar complex generative tasks benefitting from additional resources and personalization, such as dialogue systems and action planning.

Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarization

TL;DR

The work addresses salience identification and personalization in meeting summaries by proposing a three-stage RAG-based pipeline that enriches transcripts with information inferred from supplementary materials. It also adds a personalization protocol that extracts participant personas to tailor summaries for specific readers. Through the MS-AMI dataset and extensive experiments using GPT-4 Turbo and smaller backbones, the approach demonstrates improved informativeness, relevance, and reduced hallucinations for general and personalized summaries, while exploring practical deployment trade-offs. The findings highlight the value of distributing multi-source challenges across dedicated modules and pave the way for on-device and cost-aware implementations in real-world settings.

Abstract

Meeting summarization is crucial in digital communication, but existing solutions struggle with salience identification to generate personalized, workable summaries, and context understanding to fully comprehend the meetings' content. Previous attempts to address these issues by considering related supplementary resources (e.g., presentation slides) alongside transcripts are hindered by models' limited context sizes and handling the additional complexities of the multi-source tasks, such as identifying relevant information in additional files and seamlessly aligning it with the meeting content. This work explores multi-source meeting summarization considering supplementary materials through a three-stage large language model approach: identifying transcript passages needing additional context, inferring relevant details from supplementary materials and inserting them into the transcript, and generating a summary from this enriched transcript. Our multi-source approach enhances model understanding, increasing summary relevance by ~9% and producing more content-rich outputs. We introduce a personalization protocol that extracts participant characteristics and tailors summaries accordingly, improving informativeness by ~10%. This work further provides insights on performance-cost trade-offs across four leading model families, including edge-device capable options. Our approach can be extended to similar complex generative tasks benefitting from additional resources and personalization, such as dialogue systems and action planning.

Paper Structure

This paper contains 33 sections, 8 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. General Summary Pipeline
  4. Personalized Summary Pipeline
  5. Dataset
  6. Experiments
  7. Setup.
  8. Evaluation.
  9. Results and discussion on the general multi-source summarization pipeline
  10. Baseline.
  11. Structured inclusion of inferred details enhances multi-source summarization quality.
  12. Results and discussion on the personalized multi-source summarization pipeline
  13. Baseline.
  14. Detailed persona inclusion improves personalization but complicates content handling.
  15. Practical Application
  16. ...and 18 more sections

Figures (8)

  • Figure 1: Overview of the three-stage summarization pipeline. Blue boxes and arrows indicate the general pipeline. Red indicates the additional personalization modules.
  • Figure 2: Example for inferred information injected as comment in squared brackets.
  • Figure 3: Example of an extracted persona for 'Project Manager'.
  • Figure 4: Gap identification prompt template.
  • Figure 5: Information inferring prompt template.
  • ...and 3 more figures