Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UP5: Unbiased Foundation Model for Fairness-aware Recommendation

Wenyue Hua, Yingqiang Ge, Shuyuan Xu, Jianchao Ji, Yongfeng Zhang

TL;DR

This work addresses an important fairness concern in LLM-based recommender systems: user-side unfairness due to implicit encoding of sensitive attributes. It introduces Counterfactually-Fair-Prompt (CFP), an adversarial, prompt-based debiasing approach, and augments it with a Prompt Mixture to support multi-attribute removal without exponential prompt growth. Through experiments on MovieLens-1M and Insurance with T5 and OpenLlama backbones, CFP demonstrates competitive recommendation quality while significantly reducing attribute leakage, outperforming both matching- and sequential-based baselines. The approach is space- and time-efficient, requiring only prefix-prompt training, and is open-sourced to enable practical adoption and further research. Key components include the per-attribute loss $L_k = \sum_u L_{rec}^k - \lambda_k \cdot L_{dis}^k$ and the PM-based combination of CFPs to handle multiple sensitive features.

Abstract

Recent advances in Foundation Models such as Large Language Models (LLMs) have propelled them to the forefront of Recommender Systems (RS). Despite their utility, there is a growing concern that LLMs might inadvertently perpetuate societal stereotypes, resulting in unfair recommendations. Since fairness is critical for RS as many users take it for decision-making and demand fulfillment, this paper focuses on user-side fairness for LLM-based recommendation where the users may require a recommender system to be fair on specific sensitive features such as gender or age. In this paper, we dive into the extent of unfairness exhibited by LLM-based recommender models based on both T5 and LLaMA backbones, and discuss appropriate methods for promoting equitable treatment of users in LLM-based recommendation models. We introduce a novel Counterfactually-Fair-Prompt (CFP) method towards Unbiased Foundation mOdels (UFO) for fairness-aware LLM-based recommendation. Experiments are conducted on two real-world datasets, MovieLens-1M and Insurance, and compared with both matching-based and sequential-based fairness-aware recommendation models. Results show that CFP achieves better recommendation performance with a high level of fairness. Data and code are open-sourced at https://github.com/agiresearch/UP5.

UP5: Unbiased Foundation Model for Fairness-aware Recommendation

TL;DR

This work addresses an important fairness concern in LLM-based recommender systems: user-side unfairness due to implicit encoding of sensitive attributes. It introduces Counterfactually-Fair-Prompt (CFP), an adversarial, prompt-based debiasing approach, and augments it with a Prompt Mixture to support multi-attribute removal without exponential prompt growth. Through experiments on MovieLens-1M and Insurance with T5 and OpenLlama backbones, CFP demonstrates competitive recommendation quality while significantly reducing attribute leakage, outperforming both matching- and sequential-based baselines. The approach is space- and time-efficient, requiring only prefix-prompt training, and is open-sourced to enable practical adoption and further research. Key components include the per-attribute loss and the PM-based combination of CFPs to handle multiple sensitive features.

Abstract

Recent advances in Foundation Models such as Large Language Models (LLMs) have propelled them to the forefront of Recommender Systems (RS). Despite their utility, there is a growing concern that LLMs might inadvertently perpetuate societal stereotypes, resulting in unfair recommendations. Since fairness is critical for RS as many users take it for decision-making and demand fulfillment, this paper focuses on user-side fairness for LLM-based recommendation where the users may require a recommender system to be fair on specific sensitive features such as gender or age. In this paper, we dive into the extent of unfairness exhibited by LLM-based recommender models based on both T5 and LLaMA backbones, and discuss appropriate methods for promoting equitable treatment of users in LLM-based recommendation models. We introduce a novel Counterfactually-Fair-Prompt (CFP) method towards Unbiased Foundation mOdels (UFO) for fairness-aware LLM-based recommendation. Experiments are conducted on two real-world datasets, MovieLens-1M and Insurance, and compared with both matching-based and sequential-based fairness-aware recommendation models. Results show that CFP achieves better recommendation performance with a high level of fairness. Data and code are open-sourced at https://github.com/agiresearch/UP5.
Paper Structure (30 sections, 6 equations, 11 figures, 15 tables, 1 algorithm)

This paper contains 30 sections, 6 equations, 11 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fairness of Recommender Systems.
  4. Fairness of Large Language Models.
  5. Preliminary of LLM-based Recommendation
  6. Motivating Fairness Concerns.
  7. Counterfactually-Fair Prompting
  8. Prompt Mixture
  9. Experiments
  10. Experimental Setup
  11. Datasets
  12. Evaluation Metrics
  13. LLM Backbone
  14. Baselines
  15. Implementation Details
  16. ...and 15 more sections

Figures (11)

  • Figure 1: Toy examples of the input-output for prompt-driven LLM-based recommendation models.
  • Figure 2: Counterfactual fairness of LLM-based recommendation given the user's choice of sensitive attribute.
  • Figure 3: Inferring sensitive attribute information from LLM-based recommendation model.
  • Figure 4: Counterfactually-Fair-Prompting method for sensitive attribute mitigation and fairness improvement
  • Figure 5: Prompt Mixture over CFPs from 3 attributes
  • ...and 6 more figures