Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system

TL;DR

This work tackles the challenge of post-meeting recap in modern hybrid work environments by proposing two complementary recaps: quick highlights for action-oriented planning and a hierarchical chapters view for detailed knowledge sharing. Implemented via two transformer-based summarization pipelines (highlights: extractive+abstractive; hierarchical: topic segmentation with chapter-level summaries and titles), the system is evaluated through in-context task-based interviews with seven Microsoft participants to understand how these recaps support personal and group work. The study reveals that both designs offer distinct benefits—highlights for quick recall and consensus-building, hierarchical recaps for comprehensive context and knowledge sharing—with user interactions (add/edit/delete) providing actionable signals to improve AI alignment. The findings yield design implications around personalization, integration of organizational artifacts, and privacy considerations, and point to future work in collecting training data from user interactions to enhance recap quality and applicability across organizational contexts.

Abstract

Meetings play a critical infrastructural role in coordinating work. The recent surge of hybrid and remote meetings in computer-mediated spaces has led to new problems (e.g., more time spent in less engaging meetings) and new opportunities (e.g., automated transcription/captioning and recap support). Advances in dialogue summarization offer the potential for improving post-meeting experiences, but fixed-length summaries often fail to meet diverse needs, such as quick overviews or detailed insights. To address these gaps, we use cognitive science and discourse theories to conceptualize two recap designs: important highlights and a structured, hierarchical minutes view, targeting complementary recap needs. We operationalize these representations into high-fidelity prototypes using dialogue summarization. Finally, we evaluate the representations' effectiveness with seven users in the context of their work meetings at Microsoft. Our results show both recap types are valuable in different contexts, enabling collaboration through discussions and consensus-building. Exploring the meaning of users adding, editing, and deleting from recaps suggests varying alignment for using these actions to improve AI-recap. Our design implications, such as incorporating organizational artifacts (e.g., linking presentations) in recaps and personalizing context, advance the discourse of effective recap designs for organizational work and support past results from cognition studies.

Figures (4)

• Figure 1: Overview of our work, highlighting the identification of salient recap designs to address information needs of meeting participants (RQ1), in-context task-based evaluation of the recap designs for personal and collaborative work using participants' real meetings (RQ2), and exploring implications of attendees' post meeting interactions with the recap for improving the AI recap (RQ3).
• Figure 2: Meeting recap UX that exemplifies design rationales
• Figure 3: We apply the text-tiling segmentation process using BERT for segmenting long sequences of meeting dialogues into individual chapters (topics) hearst-1997-text. Utterance refers to a single sentence spoken by a meeting participant in the transcript.
• Figure 4: Illustration of the modeling pipeline that we use to generate the two recap experiences (\ref{['fig:pipeline']}).