ArguMentor: Augmenting User Experiences with Counter-Perspectives

TL;DR

ArguMentor addresses the challenge of one-sided op-eds by integrating AI-generated counterarguments, inline highlighting, and context-based summaries within a two-stage system that also offers interactive features like Q&A and debate. Through a within-subjects study (N=24) across diverse op-eds, the approach increases the quantity and quality of claims and counter-arguments, boosts elements of critical thinking, and yields generally positive user experiences, albeit with time-related constraints on daily use. The work contributes a practical framework for AI-assisted critical reading, demonstrates design considerations for real-time, in-text argumentation, and discusses implications for reducing echo chambers and improving civic reasoning in everyday news consumption.

Abstract

We encounter arguments everyday in the form of social media posts, presidential debates, news articles, and even advertisements. A ubiquitous, influential example is the opinion piece (op-ed). Opinion pieces can provide valuable perspectives, but they often represent only one side of a story, which can make readers susceptible to confirmation bias and echo chambers. Exposure to different perspectives can help readers overcome these obstacles and form more robust, nuanced views on important societal issues. We designed ArguMentor, a human-AI collaboration system that highlights claims in opinion pieces, identifies counter-arguments for them using a LLM, and generates a context-based summary of based on current events. It further enhances user understanding through additional features like a Q\&A bot (that answers user questions pertaining to the text), DebateMe (an agent that users can argue any side of the piece with) and highlighting (where users can highlight a word or passage to get its definition or context). Our evaluation on news op-eds shows that participants can generate more arguments and counter-arguments and display higher critical thinking skills after engaging with the system. Further discussion highlights a more general need for this kind of a system.

Figures (7)

• Figure 1: Architecture Diagram
• Figure 3: ArguMentor System Screenshot
• Figure 4: Experiment Design and Procedure: The process always starts with the baseline condition (reading the article by itself), and ends with the experimental condition (reading the article with the system). Each person only reads two articles and submits questionnaires right after reading.
• Figure 5: Performance: Number of Claims and Counter-Arguments with and without ArguMentor.
• Figure 6: Performance: Average Critical Thinking Skills over all three domains