Do Differences in Values Influence Disagreements in Online Discussions?

TL;DR

The paper investigates whether differences in personal values explain disagreements in online discussions. It develops Value Profile Estimation (VPE) to derive per-user value profiles from large-scale Reddit data using the Schwartz value model and transformer-based value extraction, and it evaluates whether value-profile dissimilarity correlates with disagreement. It finds that value conflicts predict disagreement only under specific conditions (diverse and relevant domains) and that incorporating value information can modestly improve agreement prediction. The work demonstrates that value-aware analysis can enhance understanding and moderation of online deliberations, while highlighting noise and biases in behavior-based value estimation.

Abstract

Disagreements are common in online discussions. Disagreement may foster collaboration and improve the quality of a discussion under some conditions. Although there exist methods for recognizing disagreement, a deeper understanding of factors that influence disagreement is lacking in the literature. We investigate a hypothesis that differences in personal values are indicative of disagreement in online discussions. We show how state-of-the-art models can be used for estimating values in online discussions and how the estimated values can be aggregated into value profiles. We evaluate the estimated value profiles based on human-annotated agreement labels. We find that the dissimilarity of value profiles correlates with disagreement in specific cases. We also find that including value information in agreement prediction improves performance.

Figures (12)

• Figure 1: Setup of measuring value conflicts by means of Value Profile Estimation (VPE).
• Figure 2: Visualization of the covariance between values in estimated profiles.
• Figure 3: $BF_{10}$ scores obtained for the combinations of data, value estimation methods, and scoring metrics.
• Figure 4: $BF_{10}$ scores for all similarity scores and task instances comparing VPE and self-reported profiles.
• Figure 5: $F_1$ scores when adding extra context information. Symbols above bars show changes with respect to text-only: $--$ for $\Delta F_1 < -0.1$; $-$ for $-0.1 < \Delta F_1 < 0$; $=$ for $\Delta F_1 = 0$; and $+$ for $\Delta F_1 > 0$.