Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A More Advanced Group Polarization Measurement Approach Based on LLM-Based Agents and Graphs

Zixin Liu, Ji Zhang, Yiran Ding

TL;DR

This work tackles the challenge of measuring group polarization on social media where traditional methods struggle with scale and textual nuance. It proposes a graph-based framework built on a temporal Community Sentiment Network (CSN) and a Community Opposition Index (COI), constructed via an LLM-based multi-agent system. Key contributions include (1) a temporal CSN representation of subgroups and sentiments, (2) an LLM-driven pipeline for accurate stance detection and CSN construction, and (3) a COI metric that combines inter-subgroup sentiment with internal cohesion, computed as $COI = \sum_i(\frac{n_i}{N} t_i \; \sum_j (-e_{ij}) \cdot 1_{e_{ij} \le 0})$. Zero-shot stance-detection experiments on SEM16, P-Stance, and VAST datasets show competitive results, including an 8.4% improvement on VAST, demonstrating the approach’s practical usability and interpretability.

Abstract

Group polarization is an important research direction in social media content analysis, attracting many researchers to explore this field. Therefore, how to effectively measure group polarization has become a critical topic. Measuring group polarization on social media presents several challenges that have not yet been addressed by existing solutions. First, social media group polarization measurement involves processing vast amounts of text, which poses a significant challenge for information extraction. Second, social media texts often contain hard-to-understand content, including sarcasm, memes, and internet slang. Additionally, group polarization research focuses on holistic analysis, while texts is typically fragmented. To address these challenges, we designed a solution based on a multi-agent system and used a graph-structured Community Sentiment Network (CSN) to represent polarization states. Furthermore, we developed a metric called Community Opposition Index (COI) based on the CSN to quantify polarization. Finally, we tested our multi-agent system through a zero-shot stance detection task and achieved outstanding results. In summary, the proposed approach has significant value in terms of usability, accuracy, and interpretability.

A More Advanced Group Polarization Measurement Approach Based on LLM-Based Agents and Graphs

TL;DR

This work tackles the challenge of measuring group polarization on social media where traditional methods struggle with scale and textual nuance. It proposes a graph-based framework built on a temporal Community Sentiment Network (CSN) and a Community Opposition Index (COI), constructed via an LLM-based multi-agent system. Key contributions include (1) a temporal CSN representation of subgroups and sentiments, (2) an LLM-driven pipeline for accurate stance detection and CSN construction, and (3) a COI metric that combines inter-subgroup sentiment with internal cohesion, computed as . Zero-shot stance-detection experiments on SEM16, P-Stance, and VAST datasets show competitive results, including an 8.4% improvement on VAST, demonstrating the approach’s practical usability and interpretability.

Abstract

Group polarization is an important research direction in social media content analysis, attracting many researchers to explore this field. Therefore, how to effectively measure group polarization has become a critical topic. Measuring group polarization on social media presents several challenges that have not yet been addressed by existing solutions. First, social media group polarization measurement involves processing vast amounts of text, which poses a significant challenge for information extraction. Second, social media texts often contain hard-to-understand content, including sarcasm, memes, and internet slang. Additionally, group polarization research focuses on holistic analysis, while texts is typically fragmented. To address these challenges, we designed a solution based on a multi-agent system and used a graph-structured Community Sentiment Network (CSN) to represent polarization states. Furthermore, we developed a metric called Community Opposition Index (COI) based on the CSN to quantify polarization. Finally, we tested our multi-agent system through a zero-shot stance detection task and achieved outstanding results. In summary, the proposed approach has significant value in terms of usability, accuracy, and interpretability.

Paper Structure

This paper contains 24 sections, 3 equations, 2 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Opinion Polarization Measurement
  4. LLM-Based Agents
  5. Method
  6. Community Sentiment Network
  7. A Multi-Agent System For CSN Construction
  8. Background Mining Stage
  9. Domain Specialist
  10. Subgroup Exploration Expert
  11. Semantic Analysis Stage
  12. Social Media Veteran
  13. Linguistic Expert
  14. Sentiment Analysis Expert
  15. Polarization Assessment Stage
  16. ...and 9 more sections

Figures (2)

  • Figure 1: An example of a CSN generated by Graphviz. This CSN was generated based on comments during the Russia-Ukraine conflict.
  • Figure 2: The structure of multi-agent system for CSN construction, containing Backround Mining Stage, Semantic Analysis Stage and Polarization Assessment Stage.