Help Me Reflect: Leveraging Self-Reflection Interface Nudges to Enhance Deliberativeness on Online Deliberation Platforms

TL;DR

This paper tackles how interface-based self-reflection nudges affect deliberative quality on online platforms. It deploys two studies using a GPT-3.5-powered Help Me Reflect interface to compare five reflectors (Persona, Temporal Prompts, Analogies/Metaphors, Cultural Prompts, Storytelling) and then focuses on the top three (Persona, Temporal Prompts, Storytelling) to assess their impact on deliberativeness. The findings show that all three top reflectors enhance deliberative quality relative to a control, with Persona boosting diversity and justification, Temporal Prompts boosting diversity and personal experience sharing, and Storytelling contributing to knowledge acquisition and constructiveness. The results offer practical guidance on reflector selection and deployment to shape online discussions, suggesting that LLM-powered reflection nudges can be integrated into platforms to foster more thoughtful, balanced, and informative deliberations.

Abstract

The deliberative potential of online platforms has been widely examined. However, little is known about how various interface-based reflection nudges impact the quality of deliberation. This paper presents two user studies with 12 and 120 participants, respectively, to investigate the impacts of different reflective nudges on the quality of deliberation. In the first study, we examined five distinct reflective nudges: persona, temporal prompts, analogies and metaphors, cultural prompts and storytelling. Persona, temporal prompts, and storytelling emerged as the preferred nudges for implementation on online deliberation platforms. In the second study, we assess the impacts of these preferred reflectors more thoroughly. Results revealed a significant positive impact of these reflectors on deliberative quality. Specifically, persona promotes a deliberative environment for balanced and opinionated viewpoints while temporal prompts promote more individualised viewpoints. Our findings suggest that the choice of reflectors can significantly influence the dynamics and shape the nature of online discussions.

Figures (18)

• Figure 1: Examples of existing online deliberation platforms that integrate reflection approaches (from left): Darmawansah et al. darmawansah2022empowering implemented the Collective Reflection-based Argumentation Mapping (CR-AM) strategy on online platform, Kialo Edu. On this platform, students collaboratively respond to a specific thesis by providing pro/con claims, supporting arguments and counterarguments; ConsiderItkriplean2012supporting is a platform to support users' reflection process by guiding them to reflect on trade-offs of policies through the creation of pros and cons points; BCauseanastasiou2023bcause incorporates reflective approaches by using a pro/con argumentation structure to organize the conversation and facilitate healthy online deliberation.
• Figure 2: Study Procedure. 1: We reviewed existing literature on reflection and reflexivity approaches on self-reflection for online deliberation. 2: We excluded approaches that do not align with our selection criteria. 3. We then conducted our first user study to answer RQ1 on users' preferred reflectors for implementation on online deliberation platforms. 4. Subsequently, we conducted an experimental study to answer RQ2 on assessing the impacts of the reflectors on users' deliberativeness.
• Figure 3: Example of the Five Selected Reflectors.
• Figure 4: Key features in Help Me Reflect. 1: Users can click on the Recreate button to browse through different textual prompts under a specific reflector or 2: click on the Try New Reflector button to navigate back to the different array of reflectors to choose another reflector. Additional features are present for storytelling to allow users to 3a: track their reading process and 3b: continue with their reading on the current story.
• Figure 5: Mean ranking for the five reflectors with 1 being the lowest rank and 5 being the highest rank. Error bars show .95 confidence intervals. We report the results of the ANCOVA test and pairwise comparisons with BH correction, where * : p < .05, ** : p < .01.