Enhancing Deliberativeness: Evaluating the Impact of Multimodal Reflection Nudges

TL;DR

This work investigates how multimodal reflective nudges influence deliberation quality on online platforms. It tests two nudge types—direct persona and indirect storytelling—across four modalities (text, image, video, audio) using AI-generated content, via two studies (n=20 and n=200). Findings show that subjective modality preferences depend on the nudge type (text for persona; video for storytelling), while objective deliberativeness improves with modality alignment, particularly with video for storytelling and persona contexts. The results advocate multimodal, AI-supported reflection to enhance inclusivity and depth in online deliberation, while highlighting ethical considerations and the need for context-aware design.

Abstract

Nudging participants with text-based reflective nudges enhances deliberation quality on online deliberation platforms. The effectiveness of multimodal reflective nudges, however, remains largely unexplored. Given the multi-sensory nature of human perception, incorporating diverse modalities into self-reflection mechanisms has the potential to better support various reflective styles. This paper explores how presenting reflective nudges of different types (direct: persona and indirect: storytelling) in different modalities (text, image, video and audio) affects deliberation quality. We conducted two user studies with 20 and 200 participants respectively. The first study identifies the preferred modality for each type of reflective nudges, revealing that text is most preferred for persona and video is most preferred for storytelling. The second study assesses the impact of these modalities on deliberation quality. Our findings reveal distinct effects associated with each modality, providing valuable insights for developing more inclusive and effective online deliberation platforms.

Figures (8)

• Figure 1: Example of the Two Reflective Nudges and their Respective Four Modalities.
• Figure 2: Key features in interface. 1: Users can click on the Recreate button to browse through different variants under a specific modality or 2: click on the Exit Reflect button to navigate back to the different array of modalities to choose another modality. Additional features are present for storytelling to allow users to 3a: track their reading process and 3b: continue with their reading on the current story. 4: For audio and video formats in both direct and indirect reflective nudges, the primary difference is duration. In direct reflective nudges, audio and video average 9 to 12 seconds, while in indirect nudges, the duration extends to 90 to 120 seconds, reflecting typical short-form media content.
• Figure 3: Mean ranking for the four modalities for the direct reflective nudge (left) and indirect reflective nudge (right) with 1 being the lowest rank and 4 being the highest rank. We report the results of the ANCOVA test and pairwise comparisons with BH correction, where * : p < .05, ** : p < .01.
• Figure 4: Count of participants who shared the same feedback on the depth of self-reflection and speed of delivery for the four modalities of each reflective nudges: direct (persona) - left and indirect (storytelling) - right.
• Figure 5: Butterfly Chart for the Text Modality across Both Nudge Types