Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Games That Teach, Chats That Convince: Comparing Interactive and Static Formats for Persuasive Learning

Seyed Hossein Alavi, Zining Wang, Shruthi Chockkalingam, Raymond T. Ng, Vered Shwartz

TL;DR

A controlled user study comparing three modes of information delivery shows a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.

Abstract

Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbosity and interaction length, are more closely related to subjective experience than to actual learning. These findings highlight a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.

Games That Teach, Chats That Convince: Comparing Interactive and Static Formats for Persuasive Learning

TL;DR

A controlled user study comparing three modes of information delivery shows a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.

Abstract

Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbosity and interaction length, are more closely related to subjective experience than to actual learning. These findings highlight a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.
Paper Structure (63 sections, 2 equations, 7 figures, 18 tables)

This paper contains 63 sections, 2 equations, 7 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Educational Technology for Sustainability-Related Topics
  4. Measuring Learning in Interactive Systems
  5. Persuasive Technology for Sustainability
  6. Study Design
  7. Topics, Arguments, and Facts
  8. Experiment Conditions
  9. Essay
  10. Chatbot
  11. Text-based Game
  12. PersuLab
  13. Participants
  14. Evaluation Measures
  15. Pre-Study Questionnaire
  16. ...and 48 more sections

Figures (7)

  • Figure 1: The user interface of PersuLab, illustrating the three system variants corresponding to different assignment conditions. Each interface displays a unique User ID assigned to the participant at the top. An “End Session” button becomes active once all facts have been presented in the chat and game conditions, and after three minutes of reading in the essay condition.
  • Figure 2: System architecture and interaction workflow for PersuLab across experimental modes. The essay condition (orange) follows a single-pass content delivery pipeline using pre-generated essays that already cover all target facts. The chat and game conditions (blue) operate through an interactive loop that includes turn generation, fact coverage tracking, and memory updates. Black paths denote shared system components and data flow common to all modes (e.g., session control and logging). All interactions are logged for subsequent analysis.
  • Figure 3: Participant demographics and assignment distribution.
  • Figure 4: Mean subjective outcome ratings (5-point Likert scale) across delivery modes, aggregated across topics (n = 43). Asterisks (*) denote measures with statistically significant differences across conditions.
  • Figure 5: Stacked bar chart showing the proportion of participants experiencing perceived change across three constructs: Importance, Intention, and Belief. Each bar represents one of three experiment modes (Essay, Chat, Game), with color-coded segments. Percentages within each segment show the proportion of participants in that category.
  • ...and 2 more figures