Policies and Evaluation for Online Meeting Summarization

TL;DR

Policies and Evaluation for Online Meeting Summarization addresses online, in-meeting generation of summaries, proposing policies that decide when to read and write given a stream of dialogue. It introduces and evaluates five policies (Length-Based, Model-Based, Sliding Window, Full Rewriting, Fully Incremental) on the AutoMin dataset, using new latency and partial-sum metrics. The paper defines $EL$ (Expected Latency) to quantify update delay and $R1-AUC$ to capture intermediate-summaries quality trajectories, along with $NE$ for rewriting and $RF$ as a redundancy proxy. Results show online systems can produce high-quality final summaries comparable to offline baselines, with dynamic segmentation offering favorable quality-latency trade-offs and strong potential for practical use.

Abstract

With more and more meetings moving to a digital domain, meeting summarization has recently gained interest in both academic and commercial research. However, prior academic research focuses on meeting summarization as an offline task, performed after the meeting concludes. In this paper, we perform the first systematic study of online meeting summarization. For this purpose, we propose several policies for conducting online summarization. We discuss the unique challenges of this task compared to the offline setting and define novel metrics to evaluate latency and partial summary quality. The experiments on the AutoMin dataset show that 1) online models can produce strong summaries, 2) our metrics allow a detailed analysis of different systems' quality-latency trade-off, also taking into account intermediate outputs and 3) adaptive policies perform better than fixed scheduled ones. These findings provide a starting point for the wider research community to explore this important task.

Figures (2)

• Figure 1: Rouge curves for the best-performing system for each policy. All curves are in Appendix \ref{['app:curves']}.
• Figure 2: Rouge F1 curves for all systems