Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging Interesting Facts to Enhance User Engagement with Conversational Interfaces

Nikhita Vedula, Giuseppe Castellucci, Eugene Agichtein, Oleg Rokhlenko, Shervin Malmasi

TL;DR

Addresses sustaining user engagement in Conversational Task Assistants by injecting contextual interesting facts during dialogues. It develops a principled interestingness feature schema, constructs a cooking facts dataset of 1,379 items, and trains a multi-label classifier to identify relevant and interesting facts. It designs a dialogue policy and validates the approach with offline annotations and live testing on a multi-modal voice assistant, reporting positive user reception (66% favorable) and notable gains in satisfaction (40%) and conversation length (37%), with a small increase in task completion. The work discusses generalizability and scalability, noting limitations of manual annotation and offline-LMM approaches, and suggests retrieval-augmented LLMs as a path to end-to-end automation while ensuring factual accuracy. The results support the practical value of fact-enriched CTAs for complex guided tasks.

Abstract

Conversational Task Assistants (CTAs) guide users in performing a multitude of activities, such as making recipes. However, ensuring that interactions remain engaging, interesting, and enjoyable for CTA users is not trivial, especially for time-consuming or challenging tasks. Grounded in psychological theories of human interest, we propose to engage users with contextual and interesting statements or facts during interactions with a multi-modal CTA, to reduce fatigue and task abandonment before a task is complete. To operationalize this idea, we train a high-performing classifier (82% F1-score) to automatically identify relevant and interesting facts for users. We use it to create an annotated dataset of task-specific interesting facts for the domain of cooking. Finally, we design and validate a dialogue policy to incorporate the identified relevant and interesting facts into a conversation, to improve user engagement and task completion. Live testing on a leading multi-modal voice assistant shows that 66% of the presented facts were received positively, leading to a 40% gain in the user satisfaction rating, and a 37% increase in conversation length. These findings emphasize that strategically incorporating interesting facts into the CTA experience can promote real-world user participation for guided task interactions.

Leveraging Interesting Facts to Enhance User Engagement with Conversational Interfaces

TL;DR

Addresses sustaining user engagement in Conversational Task Assistants by injecting contextual interesting facts during dialogues. It develops a principled interestingness feature schema, constructs a cooking facts dataset of 1,379 items, and trains a multi-label classifier to identify relevant and interesting facts. It designs a dialogue policy and validates the approach with offline annotations and live testing on a multi-modal voice assistant, reporting positive user reception (66% favorable) and notable gains in satisfaction (40%) and conversation length (37%), with a small increase in task completion. The work discusses generalizability and scalability, noting limitations of manual annotation and offline-LMM approaches, and suggests retrieval-augmented LLMs as a path to end-to-end automation while ensuring factual accuracy. The results support the practical value of fact-enriched CTAs for complex guided tasks.

Abstract

Conversational Task Assistants (CTAs) guide users in performing a multitude of activities, such as making recipes. However, ensuring that interactions remain engaging, interesting, and enjoyable for CTA users is not trivial, especially for time-consuming or challenging tasks. Grounded in psychological theories of human interest, we propose to engage users with contextual and interesting statements or facts during interactions with a multi-modal CTA, to reduce fatigue and task abandonment before a task is complete. To operationalize this idea, we train a high-performing classifier (82% F1-score) to automatically identify relevant and interesting facts for users. We use it to create an annotated dataset of task-specific interesting facts for the domain of cooking. Finally, we design and validate a dialogue policy to incorporate the identified relevant and interesting facts into a conversation, to improve user engagement and task completion. Live testing on a leading multi-modal voice assistant shows that 66% of the presented facts were received positively, leading to a 40% gain in the user satisfaction rating, and a 37% increase in conversation length. These findings emphasize that strategically incorporating interesting facts into the CTA experience can promote real-world user participation for guided task interactions.
Paper Structure (26 sections, 1 figure, 3 tables, 2 algorithms)

This paper contains 26 sections, 1 figure, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Interest and Interestingness:
  4. Conversational Task Assistants:
  5. Boosting CTA Engagement with Facts
  6. Feature Schema for Interestingness
  7. Interesting Facts Dataset
  8. Incorporating Interesting Facts into Conversations
  9. Experiments and Results
  10. Interesting Fact Quality Evaluation
  11. Relevance and Interestingness Detection:
  12. Evaluating LLMs for Annotation:
  13. Interesting Facts and User Engagement:
  14. Conversation Quality Evaluation
  15. Interesting Fact Placement in Conversations:
  16. ...and 11 more sections

Figures (1)

  • Figure 1: Conversations $C_1$ and $C_2$ between two users and our CTA. $C_1$ shows an interesting fact during search. $C_2$ has two interesting facts during task execution.