CommSense: Facilitating Bias-Aware and Reflective Navigation of Online Comments for Rational Judgment

TL;DR

Online comment presentation can bias judgment through anchoring and framing; this work investigates interface-level interventions to sustain rational judgment. Through three studies, it identifies a four-stage decision path and four design requirements, then implements CommSense with a Topic Corpus Overview, Comment Navigation Panel, and Synthesis Board to support pre-engagement framing, organization, reflective prompts, and dynamic synthesis. A between-subject study shows CommSense enhances bias awareness and reflective reasoning while preserving usability and lowering cognitive load. The findings demonstrate practical, generalizable approaches to promote bias-aware sensemaking in unstructured online discourse and suggest avenues for extending these methods to health, consumer reviews, and political discussions.

Abstract

Online comments significantly influence users' judgments, yet their presentation, often determined by platform algorithms, can introduce biases, such as anchoring effects, which distort reasoning. While existing research emphasizes mitigating individual cognitive biases, the evolution of user judgments during comment engagement remains overlooked. This study investigates how presentation cues impact reasoning and explores interface design strategies to mitigate bias. Through a preliminary experiment (N=18) and a co-design workshop, we identified key challenges users face across a four-stage process and distilled four design requirements: pre-engagement framing, interactive organization, reflective prompts, and synthesis support. Based on these insights, we developed CommSense, an on-the-fly plugin that enhances user engagement with online comments by providing visual overviews and lightweight prompts to guide reasoning. A between-subject evaluation (N=24) demonstrates that CommSense improves bias awareness and reflective thinking, helping users produce more comprehensive, evidence-based rationales while maintaining high usability.

Figures (11)

• Figure 1: The study began by having participants evaluate a hotel in Istanbul based on user comments under three conditions, thinking aloud and rating it on a 1–5 scale. Post-task interviews and interaction logs were analyzed using statistical methods and reflexive thematic analysis, with results validated by a third researcher.
• Figure 2: Experimental Probe. Participants first viewed the (a) Hotel Overview Panel, then browsed user reviews in the (b) Review Browsing Interface, and rated six hotel attributes in the Attribute Rating Panel (c) based on the review content. The interface supported iterative evaluation, allowing participants to revise their ratings as their impressions evolved during the review browsing process.
• Figure 3: The decision path progresses through four stages: 1) Initial Framing, followed by an iterative cycle of 2) Evidence Foraging and 3) Belief Updating, and concluding with 4) Synthesis and Judgment.
• Figure 4: The four-stage decision paths of all 18 participants over time, grouped by experimental condition. Each horizontal bar represents one participant’s session, with colored segments indicating the time spent in each stage. The numbers in parentheses denote the frequency of score adjustments during Stages 2 and 3.
• Figure 5: Average rating trajectories across the three conditions. Lines represent the mean rating given by participants after viewing each comment, with shaded areas indicating $\pm 1$ standard deviation across participants. Positive-First (orange) starts higher, Negative-First (blue) starts lower, and Interleaved (green) shows moderate, less variable ratings. The x-axis indicates the number of comments viewed (1–10), and the y-axis shows the 1--5 rating scale.