Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants

Chen Chen, Khalil Mrini, Kemeberly Charles, Ella T. Lifset, Michael Hogarth, Alison A. Moore, Nadir Weibel, Emilia Farcas

TL;DR

EMA data collection in real-world settings faces user burden, especially when integrating voice-first IVAs. The authors introduce a unified metadata schema that models EMA questions, contextual rules, scheduling, and multimodal outputs to enable run-time rendering and rapid prototyping without modifying source code. They implement an end-to-end platform on Alexa with DynamoDB, showcasing cross-device deployment and conditional branching for adaptive question flows. Key contributions include a header-friendly data model for topics, questions, outputs, conditions, schedules, and answer validation, plus runtime rule rendering via reflection. The work offers a practical pathway to faster, more scalable voice-based EMA prototypes, potentially improving engagement and data quality in ecological studies.

Abstract

Ecological momentary assessment (EMA) is used to evaluate subjects' behaviors and moods in their natural environments, yet collecting real-time and self-report data with EMA is challenging due to user burden. Integrating voice into EMA data collection platforms through today's intelligent virtual assistants (IVAs) is promising due to hands-free and eye-free nature. However, efficiently managing conversations and EMAs is non-trivial and time consuming due to the ambiguity of the voice input. We approach this problem by rethinking the data modeling of EMA questions and what is needed to deploy them on voice-first user interfaces. We propose a unified metadata schema that models EMA questions and the necessary attributes to effectively and efficiently integrate voice as a new EMA modality. Our schema allows user experience researchers to write simple rules that can be rendered at run-time, instead of having to edit the source code. We showcase an example EMA survey implemented with our schema, which can run on multiple voice-only and voice-first devices. We believe that our work will accelerate the iterative prototyping and design process of real-world voice-based EMA data collection platforms.

Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants

TL;DR

EMA data collection in real-world settings faces user burden, especially when integrating voice-first IVAs. The authors introduce a unified metadata schema that models EMA questions, contextual rules, scheduling, and multimodal outputs to enable run-time rendering and rapid prototyping without modifying source code. They implement an end-to-end platform on Alexa with DynamoDB, showcasing cross-device deployment and conditional branching for adaptive question flows. Key contributions include a header-friendly data model for topics, questions, outputs, conditions, schedules, and answer validation, plus runtime rule rendering via reflection. The work offers a practical pathway to faster, more scalable voice-based EMA prototypes, potentially improving engagement and data quality in ecological studies.

Abstract

Ecological momentary assessment (EMA) is used to evaluate subjects' behaviors and moods in their natural environments, yet collecting real-time and self-report data with EMA is challenging due to user burden. Integrating voice into EMA data collection platforms through today's intelligent virtual assistants (IVAs) is promising due to hands-free and eye-free nature. However, efficiently managing conversations and EMAs is non-trivial and time consuming due to the ambiguity of the voice input. We approach this problem by rethinking the data modeling of EMA questions and what is needed to deploy them on voice-first user interfaces. We propose a unified metadata schema that models EMA questions and the necessary attributes to effectively and efficiently integrate voice as a new EMA modality. Our schema allows user experience researchers to write simple rules that can be rendered at run-time, instead of having to edit the source code. We showcase an example EMA survey implemented with our schema, which can run on multiple voice-only and voice-first devices. We believe that our work will accelerate the iterative prototyping and design process of real-world voice-based EMA data collection platforms.
Paper Structure (9 sections, 6 figures)

This paper contains 9 sections, 6 figures.

Table of Contents

  1. introduction
  2. Schema Design
  3. Entity Relationships and Cross-Platform Support
  4. Contextual Awareness and Conditions
  5. Schedule and Occurrence
  6. Answer Validation and Error Prompt
  7. Example Applications
  8. Conclusion
  9. Acknowledgements

Figures (6)

  • Figure 1: Example system that uses our designed schema to store and render EMA questionnaires. To try out different conversation flows during the iterative design process, UX researchers only need to modify the content in the database.
  • Figure 2: A simplified entity relationships diagram for EMA questions and the necessary attributes. As a running example, we used the types supported by DynamoDB TypeAmazonDB. Notably, the property fields in Audio_Output and Visual_Output collections vary among types of questions.
  • Figure 3: Modelling of conditional branching using a decision tree graph.
  • Figure 4: Example of how to instantiate an Answer in our metadata schema.
  • Figure 5: Example EMA survey for evaluating sedentary behavior and physical activity revised from Maher2018.
  • ...and 1 more figures