Using RL to Identify Divisive Perspectives Improves LLMs Abilities to Identify Communities on Social Media

TL;DR

This work tackles the problem of identifying user information communities on social media under emergent events where test data differ from training data. It introduces a two-LM framework in which a smaller prompt-generating LM is trained via supervised learning and reinforcement learning to produce focus_area prompts that guide a fixed LLM_task to form meaningful communities, improving downstream tasks such as bot detection and news source profiling. Four reward signals plus curriculum learning drive the RL training, yielding consistent gains across Reddit and TwiBot datasets and improved inductive profiling of news sources. The approach generalizes across LLM backbones (ChatGPT, Llama 2) and demonstrates practical impact by enhancing community-aware downstream analytics while highlighting considerations for safe, ethical deployment.

Abstract

The large scale usage of social media, combined with its significant impact, has made it increasingly important to understand it. In particular, identifying user communities, can be helpful for many downstream tasks. However, particularly when models are trained on past data and tested on future, doing this is difficult. In this paper, we hypothesize to take advantage of Large Language Models (LLMs), to better identify user communities. Due to the fact that many LLMs, such as ChatGPT, are fixed and must be treated as black-boxes, we propose an approach to better prompt them, by training a smaller LLM to do this. We devise strategies to train this smaller model, showing how it can improve the larger LLMs ability to detect communities. Experimental results show improvements on Reddit and Twitter data, on the tasks of community detection, bot detection, and news media profiling.

Figures (8)

• Figure 1: An example of the LLM Community Detection Task: Given a set of users and their textual descriptions, determine which users are similar and have similar perspective.
• Figure 2: An example of how Focus Areas can help. Without them (a), the LLM incorrectly forms the community (red users), but with them (b), the LLM focuses on the divisive issues and correctly forms the community (green).
• Figure 3: How we prompt ChatGPT to generate more informative focus areas (positive class), given ones from the training set (negative class). We then train a binary LR model on this data.
• Figure 4: An example of the prompt we used to determine the user summary. For Twitter users, based on their bio, meta-data, and tweets, we create a summary. For Reddit, we use their comments.
• Figure 5: An example of the prompt we used to generate gold focus areas. Given a set of six users, we ask $\text{LLM}_{\text{task}}$ what makes the first three users part of the same community. We also add additional instructions to the prompt to make sure that the LLM responds only with focus areas, not extra information such as user ordering.