Incorporating Different Verbal Cues to Improve Text-Based Computer-Delivered Health Messaging

TL;DR

This thesis investigates how verbal cues in text-based computer-delivered health messaging affect user engagement and behavior. It presents three empirical strands: (i) Directed Diversity to generate diverse, efficacious health messages via crowd prompts; (ii) how chatbot language formality influences user disclosure and the quality of open-text responses; and (iii) how formats for referencing past user utterances impact perceived intelligence, engagement, and privacy. The findings show that diversity-promoting prompting can enhance ideation diversity while maintaining usefulness, formal style can improve perceived competence and information quality in health contexts, and explicit past-utterance referencing boosts perceived intelligence but raises privacy concerns. Collectively, the work informs scalable design of empathetic, user-aware digital health interventions that balance engagement with privacy considerations, and discusses the potential of LLMs versus crowd-based generation for future health messaging systems.

Abstract

The ubiquity of smartphones has led to an increase in on demand healthcare being supplied. For example, people can share their illness-related experiences with others similar to themselves, and healthcare experts can offer advice for better treatment and care for remediable, terminal and mental illnesses. As well as this human-to-human communication, there has been an increased use of human-to-computer digital health messaging, such as chatbots. These can prove advantageous as they offer synchronous and anonymous feedback without the need for a human conversational partner. However, there are many subtleties involved in human conversation that a computer agent may not properly exhibit. For example, there are various conversational styles, etiquettes, politeness strategies or empathic responses that need to be chosen appropriately for the conversation. Encouragingly, computers are social actors (CASA) posits that people apply the same social norms to computers as they would do to people. On from this, previous studies have focused on applying conversational strategies to computer agents to make them embody more favourable human characteristics. However, if a computer agent fails in this regard it can lead to negative reactions from users. Therefore, in this dissertation we describe a series of studies we carried out to lead to more effective human-to-computer digital health messaging. In our first study, we use the crowd [...] Our second study investigates the effect of a health chatbot's conversational style [...] In our final study, we investigate the format used by a chatbot when [...] In summary, we have researched how to create more effective digital health interventions starting from generating health messages, to choosing an appropriate formality of messaging, and finally to formatting messages which reference a user's previous utterances.

Figures (82)

• Figure 1: Pipeline of the overall technical approach to extract, embed, and select phrases to generate diverse prompts. a) Phrase extraction by collecting phrases from online articles and discussion forums (shown as pages), filtering phrases to select a clean subset (shown as the black dash for each phrase); b) Phrase embedding using the Universal Sentence Encoder cer2018universal to compute the embedding vector of each phrase (shown as scatter plot); c) Phrase Selection by constructing the minimal spanning tree to select optimally spaced phrases (see Figure \ref{['fig:Directed-Fig2']} for more details).
• Figure 2: Demonstration of pairwise embedding angular distances between an example text items (first data row) and neighboring text items. Text items with semantically similar words have smaller distances. For interpretability, we highlighted words to indicate darker color with higher cosine similarity to the first phrase.
• Figure 3: Procedure to direct ideation towards diverse phrases (top) and away from prior or redundant ideas (bottom). To attract ideation with diverse prompts: a) start with embeddings of corpus-extracted phrases; b) construct minimum spanning tree (MST); c) traverse tree to select distant prompts from clusters (most distant points as green dots, in clustered phrases as green ellipses); d) selected prompts are the most diverse. To repel ideation from prior ideas, e) compute embeddings of prior ideas (red hollow dots); f) compute prompt-ideation pairwise distances of all prompts from each prior ideation, exclude phrases (dotted black circles) with pairwise distance less than a user- defined threshold (red bubble), and construct the MST with remaining phrases; g) traverse MST to select a user- defined number of prompts; h) selected prompts are diverse, yet avoids prior ideas.
• Figure 4: Diversity prompting evaluation framework to evaluate prompting to support diverse ideation along the ideation chain. We pose research questions (RQ3.1-3.3) between each step to validate the ideation diversification process. For each step, we manipulate or measure various experiment constructs to track how well ideators are prompted to generate creative ideas. Except for prompt selection, each construct refers to a statistical factor determined from factor analyses of multiple dependent variables. Constructs are grouped in colored blocks indicating different data collection methods as follows: (Blue - Computed embedding-based metric, Green - ratings from ideators, Red - ratings from validators, Purple - thematic coding of ideations).
• Figure 5: Metrics of distances between two points in a multi-dimensional vector space. Each metric can be calculated for an individual text item. These metrics can apply to the embedding of phrases or ideations.