Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms

Rajvardhan Oak, Muhammad Haroon, Claire Jo, Magdalena Wojcieszak, Anshuman Chhabra

TL;DR

This work tackles exposure to harmful content in platform recommendations by proposing an LLM-based re-ranking framework that uses preferential pairwise ranking with constraint prompts. It introduces $TP_k$, $PP_k$, and $EWN$ as metrics to evaluate how well the re-ranking minimizes harm while preserving useful content, and demonstrates strong gains over traditional moderation baselines across three harm datasets and multiple LLMs in zero-shot and few-shot settings. The results show that the approach generalizes across harm types and remains robust to varying harm ratios and model choices, including open-source LLMs, offering a scalable, data-efficient alternative to labeled moderation. The study highlights practical implications for safer content delivery and notes future directions such as incorporating multi-modal inputs and extending to other ranking-driven safety goals.

Abstract

Social media platforms utilize Machine Learning (ML) and Artificial Intelligence (AI) powered recommendation algorithms to maximize user engagement, which can result in inadvertent exposure to harmful content. Current moderation efforts, reliant on classifiers trained with extensive human-annotated data, struggle with scalability and adapting to new forms of harm. To address these challenges, we propose a novel re-ranking approach using Large Language Models (LLMs) in zero-shot and few-shot settings. Our method dynamically assesses and re-ranks content sequences, effectively mitigating harmful content exposure without requiring extensive labeled data. Alongside traditional ranking metrics, we also introduce two new metrics to evaluate the effectiveness of re-ranking in reducing exposure to harmful content. Through experiments on three datasets, three models and across three configurations, we demonstrate that our LLM-based approach significantly outperforms existing proprietary moderation approaches, offering a scalable and adaptable solution for harm mitigation.

Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms

TL;DR

This work tackles exposure to harmful content in platform recommendations by proposing an LLM-based re-ranking framework that uses preferential pairwise ranking with constraint prompts. It introduces , , and as metrics to evaluate how well the re-ranking minimizes harm while preserving useful content, and demonstrates strong gains over traditional moderation baselines across three harm datasets and multiple LLMs in zero-shot and few-shot settings. The results show that the approach generalizes across harm types and remains robust to varying harm ratios and model choices, including open-source LLMs, offering a scalable, data-efficient alternative to labeled moderation. The study highlights practical implications for safer content delivery and notes future directions such as incorporating multi-modal inputs and extending to other ranking-driven safety goals.

Abstract

Social media platforms utilize Machine Learning (ML) and Artificial Intelligence (AI) powered recommendation algorithms to maximize user engagement, which can result in inadvertent exposure to harmful content. Current moderation efforts, reliant on classifiers trained with extensive human-annotated data, struggle with scalability and adapting to new forms of harm. To address these challenges, we propose a novel re-ranking approach using Large Language Models (LLMs) in zero-shot and few-shot settings. Our method dynamically assesses and re-ranks content sequences, effectively mitigating harmful content exposure without requiring extensive labeled data. Alongside traditional ranking metrics, we also introduce two new metrics to evaluate the effectiveness of re-ranking in reducing exposure to harmful content. Through experiments on three datasets, three models and across three configurations, we demonstrate that our LLM-based approach significantly outperforms existing proprietary moderation approaches, offering a scalable and adaptable solution for harm mitigation.
Paper Structure (43 sections, 10 equations, 4 figures, 11 tables, 1 algorithm)

This paper contains 43 sections, 10 equations, 4 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Recommendation-Driven Harms
  4. Harm Mitigation
  5. Mitigating Harmful Content Exposure Using LLMs and Re-ranking
  6. Problem Formulation
  7. Preferential Pairwise Ranking
  8. Specifying Preference Constraints $\mathcal{C}$
  9. In-Context Learning (ICL)
  10. Evaluating Re-rankings
  11. Experimental Setup
  12. Datasets
  13. Content Sequences
  14. Baselines
  15. Implementation Details
  16. ...and 28 more sections

Figures (4)

  • Figure 1: An overview of our re-ranking approach for mitigating exposure to harmful content. We prompt the LLM with the input set of recommendations and a set of preference constraints. The LLM re-ranks the the recommendations in accordance with the provided preferences. Here, recommendations B, E, and G are harmful and hence, downranked.
  • Figure 2: Performance of our proposed method with different preference constraint strategies on varying ratios of harmful content. Higher values indicate better alignment with less harmful content exposure. Our approaches outperform all baselines by a wide margin for the TP$k$ (A, B), PP$k$ (D-F) and EWN (C) metrics. Note that as the harm ratio increases in content sequences, exposure to harmful content increases as well.
  • Figure 3: Performance of ICL with varying number of exemplars, as measured by our defined metrics. Harm mitigation effectiveness does not meaningfully improve by increasing the number of exemplars.
  • Figure 4: EWN values for Llama2, Mistral, and GPT-3.5-Turbo.