Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ArticulatePro: A Comparative Study on a Proactive and Non-Proactive Assistant in a Climate Data Exploration Task

Roderick Tabalba, Christopher J. Lee, Giorgio Tran, Nurit Kirshenbaum, Jason Leigh

TL;DR

This study developed a digital assistant that continuously listens to conversations and proactively generates relevant visualizations during data exploration tasks and identifies key challenges in its implementation, offering insights for future research.

Abstract

Recent advances in Natural Language Interfaces (NLIs) and Large Language Models (LLMs) have transformed our approach to NLP tasks, shifting the focus towards a more Pragmatics-based approach. This shift enables more natural interactions between humans and voice assistants, which have been historically difficult to achieve. Pragmatics involves understanding how users often talk out of turn, interrupt one another, or provide relevant information without being explicitly asked (maxim of quantity). To explore this, we developed a digital assistant that continuously listens to conversations and proactively generates relevant visualizations during data exploration tasks. In a within-subject study, participants interacted with both proactive and non-proactive versions of a voice assistant while exploring the Hawaii Climate Data Portal (HCDP). Results suggest that the proactive assistant enhanced user engagement and facilitated quicker insights. Our study highlights the potential of Pragmatic, proactive AI in NLIs and identifies key challenges in its implementation, offering insights for future research.

ArticulatePro: A Comparative Study on a Proactive and Non-Proactive Assistant in a Climate Data Exploration Task

TL;DR

This study developed a digital assistant that continuously listens to conversations and proactively generates relevant visualizations during data exploration tasks and identifies key challenges in its implementation, offering insights for future research.

Abstract

Recent advances in Natural Language Interfaces (NLIs) and Large Language Models (LLMs) have transformed our approach to NLP tasks, shifting the focus towards a more Pragmatics-based approach. This shift enables more natural interactions between humans and voice assistants, which have been historically difficult to achieve. Pragmatics involves understanding how users often talk out of turn, interrupt one another, or provide relevant information without being explicitly asked (maxim of quantity). To explore this, we developed a digital assistant that continuously listens to conversations and proactively generates relevant visualizations during data exploration tasks. In a within-subject study, participants interacted with both proactive and non-proactive versions of a voice assistant while exploring the Hawaii Climate Data Portal (HCDP). Results suggest that the proactive assistant enhanced user engagement and facilitated quicker insights. Our study highlights the potential of Pragmatic, proactive AI in NLIs and identifies key challenges in its implementation, offering insights for future research.
Paper Structure (39 sections, 7 figures, 1 table)

This paper contains 39 sections, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Relevant Works
  3. Natural Language for Data Exploration
  4. Proactive Applications
  5. Pragmatics
  6. ArticulatePro
  7. An Overview of the System
  8. Investigating Proactive Opportunities
  9. Pragmatic Approach for Proactive Opportunities
  10. Technical Overview
  11. User Interaction
  12. Speech
  13. History
  14. Query Refinement
  15. Neural Network Training
  16. ...and 24 more sections

Figures (7)

  • Figure 1: (1) General workspace for moving, resizing, and deleting selected visualizations. (2) The digital persona of ArticulatePro that creates visualizations. (3) A visualization conveyor belt (inspired by a sushi conveyor belt) that displays generated visualizations from the digital assistant. The assistant creates visualizations and displays them on the conveyor belt, where users can select charts to move them to the general workspace for further analysis.
  • Figure 2: The architectural design of ArticulatePro. (1) User interaction consists of the user's speech and interaction of charts. (2) Speech module transcribes the user's speech using OpenAI's Whisper model radford2023robust. (3) History consists of a history of the user's utterance and chart interaction. (4) Query Refinement creates a succinct query based on the history component. (5) Chart Reasoner extracts and decides what chart to construct and what attributes to visualize (6) Presentation component for chart construction and response generator.
  • Figure 3: This chart displays the total number of utterances for each group. The chart is separated between the two groups P (Arti) first and NP (Marti) first. In almost every session but 2, users talked more with Arti.
  • Figure 4: This chart displays the total number of utterances for each group. The chart is separated between the two groups P (Arti) first and NP (Marti) first. Similarly in the total utterance section, users used more task-relevant keywords in almost every session but session 2.
  • Figure 5: This chart displays the total number of discoveries for each group. The chart is separated between the two groups P (Arti) first and NP (Marti) first. In almost all sessions but session 2, the participants made more discoveries with Arti than with Marti.
  • ...and 2 more figures