Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Introducing MeMo: A Multimodal Dataset for Memory Modelling in Multiparty Conversations

Maria Tsfasman, Bernd Dudzik, Kristian Fenech, Andras Lorincz, Catholijn M. Jonker, Catharine Oertel

TL;DR

The MeMo corpus is introduced, the first conversational dataset annotated with participants' memory retention reports, aimed at facilitating computational modelling of human conversational memory.

Abstract

Conversational memory is the process by which humans encode, retain and retrieve verbal, non-verbal and contextual information from a conversation. Since human memory is selective, differing recollections of the same events can lead to misunderstandings and misalignments within a group. Yet, conversational facilitation systems, aimed at advancing the quality of group interactions, usually focus on tracking users' states within an individual session, ignoring what remains in each participant's memory after the interaction. Understanding conversational memory can be used as a source of information on the long-term development of social connections within a group. This paper introduces the MeMo corpus, the first conversational dataset annotated with participants' memory retention reports, aimed at facilitating computational modelling of human conversational memory. The MeMo corpus includes 31 hours of small-group discussions on Covid-19, repeated 3 times over the term of 2 weeks. It integrates validated behavioural and perceptual measures, audio, video, and multimodal annotations, offering a valuable resource for studying and modelling conversational memory and group dynamics. By introducing the MeMo corpus, analysing its validity, and demonstrating its usefulness for future research, this paper aims to pave the way for future research in conversational memory modelling for intelligent system development.

Introducing MeMo: A Multimodal Dataset for Memory Modelling in Multiparty Conversations

TL;DR

The MeMo corpus is introduced, the first conversational dataset annotated with participants' memory retention reports, aimed at facilitating computational modelling of human conversational memory.

Abstract

Conversational memory is the process by which humans encode, retain and retrieve verbal, non-verbal and contextual information from a conversation. Since human memory is selective, differing recollections of the same events can lead to misunderstandings and misalignments within a group. Yet, conversational facilitation systems, aimed at advancing the quality of group interactions, usually focus on tracking users' states within an individual session, ignoring what remains in each participant's memory after the interaction. Understanding conversational memory can be used as a source of information on the long-term development of social connections within a group. This paper introduces the MeMo corpus, the first conversational dataset annotated with participants' memory retention reports, aimed at facilitating computational modelling of human conversational memory. The MeMo corpus includes 31 hours of small-group discussions on Covid-19, repeated 3 times over the term of 2 weeks. It integrates validated behavioural and perceptual measures, audio, video, and multimodal annotations, offering a valuable resource for studying and modelling conversational memory and group dynamics. By introducing the MeMo corpus, analysing its validity, and demonstrating its usefulness for future research, this paper aims to pave the way for future research in conversational memory modelling for intelligent system development.
Paper Structure (58 sections, 4 figures, 8 tables)

This paper contains 58 sections, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Motivation for creating MeMo
  3. Guiding principles for collecting MeMo
  4. P1: Maximising ecological validity
  5. P2: Maximising the construct validity of conversational memory measure
  6. P3: Considering Context-sensitivity of Memory Processes
  7. Related work
  8. Facilitated Modelling Perspective
  9. Memory process
  10. Recorded behaviour & measures
  11. Task context & memory task
  12. Sample representation
  13. Data collection procedure
  14. Overall procedure
  15. Ethical approval.
  16. ...and 43 more sections

Figures (4)

  • Figure 1: Experimental set-up. Upper flowchart - overall set-up. In the lower part - illustration of the procedure for every group session. On the second screen, there is a screenshot of a discussion from the MeMo corpus, except for the 4th participant shown with a person icon. The phrase the icon person produces is made-up but matches the conversations included in MeMo.
  • Figure 2: MeMo corpus processing and curation steps
  • Figure 3: Duration and word-count distribution of memorable moments with examples of memorable moments shown with points and full reports on the right.
  • Figure 4: The change in perceived social distance between participants throughout the 3 sessions of the interactions, reported through IOS scale, with 1 = no overlap, 3 = some overlap and 7 = most overlap aron1992IOS.