Filtering Discomforting Recommendations with Large Language Models

TL;DR

This work tackles the subjective problem of discomforting recommendations by introducing DiscomfortFilter, an LLM-based tool that constructs an editable, feature-level user preference profile from interaction data and supports conversational rule configuration to filter content. The approach emphasizes transparency and contestability by providing filtering logs and explanations, while remaining plug-and-play so it can operate atop existing recommender outputs without modifying platform models. Evaluation includes offline proxy tasks showing improved LLM reasoning and open-source models rivaling some commercial equivalents, plus a one-week Zhihu user study (n=24) demonstrating usability, perceived usefulness, and a measurable reduction in exposure to discomforting content. The findings advance human-centered design in recommender systems and discuss practical impacts, challenges with LLM alignment, and directions for broader deployment and multi-modal extension.

Abstract

Personalized algorithms can inadvertently expose users to discomforting recommendations, potentially triggering negative consequences. The subjectivity of discomfort and the black-box nature of these algorithms make it challenging to effectively identify and filter such content. To address this, we first conducted a formative study to understand users' practices and expectations regarding discomforting recommendation filtering. Then, we designed a Large Language Model (LLM)-based tool named DiscomfortFilter, which constructs an editable preference profile for a user and helps the user express filtering needs through conversation to mask discomforting preferences within the profile. Based on the edited profile, DiscomfortFilter facilitates the discomforting recommendations filtering in a plug-and-play manner, maintaining flexibility and transparency. The constructed preference profile improves LLM reasoning and simplifies user alignment, enabling a 3.8B open-source LLM to rival top commercial models in an offline proxy task. A one-week user study with 24 participants demonstrated the effectiveness of DiscomfortFilter, while also highlighting its potential impact on platform recommendation outcomes. We conclude by discussing the ongoing challenges, highlighting its relevance to broader research, assessing stakeholder impact, and outlining future research directions.

Figures (12)

• Figure 1: The workflow of DiscomfortFilter.
• Figure 2: Problem formulation.
• Figure 3: The process of presenting items to a user before introducing DiscomfortFilter.
• Figure 4: Detailed design of DiscomfortFilter.
• Figure 5: The Preference Profile Construction Module is a multi-agent pipeline powered by LLMs.