Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Fair Large Language Model-based Recommender Systems without Costly Retraining

Jin Li, Huilin Gu, Shoujin Wang, Qi Zhang, Shui Yu, Chen Wang, Xiwei Xu, Fang Chen

TL;DR

This work tackles fairness in large language model based recommender systems by addressing two key obstacles: lack of generality across bias types and prohibitive retraining costs. It introduces FUDLR, a Fast Unified Debiasing framework that reinterprets debiasing as machine unlearning in two stages: (1) Bias Identification via a learnable mask that selects bias-inducing samples by optimizing a multi-objective loss balancing fairness, accuracy, and sparsity, and (2) Debiasing via Unlearning that uses influence functions to approximate a retrained model with a sparse subset removed, computing $\Delta \theta \approx \frac{1}{n} \mathbf{H}_{\theta}^{-1} \sum_{z_k \in \mathcal{D}_{\text{unlearn}}} \nabla_{\theta} \mathcal{L}_{\rm LLM}(z_k; \theta)$. The framework generalizes to multiple bias types through differentiable fairness metrics, leverages LoRA adapters and Hessian-vector products for efficiency, and achieves substantial fairness improvements with minimal accuracy loss and large speedups across real-world datasets. Extensive experiments on popularity and attribute biases demonstrate superior trade-offs between fairness and utility and significant reductions in computational cost compared with retraining-based baselines. The work offers a practical path toward socially responsible LLM-RS and opens avenues for refined, personalized debiasing in large-scale systems.

Abstract

Large Language Models (LLMs) have revolutionized Recommender Systems (RS) through advanced generative user modeling. However, LLM-based RS (LLM-RS) often inadvertently perpetuates bias present in the training data, leading to severe fairness issues. Addressing these fairness problems in LLM-RS faces two significant challenges. 1) Existing debiasing methods, designed for specific bias types, lack the generality to handle diverse or emerging biases in real-world applications. 2) Debiasing methods relying on retraining are computationally infeasible given the massive parameter scale of LLMs. To overcome these challenges, we propose FUDLR (Fast Unified Debiasing for LLM-RS). The core idea is to reformulate the debiasing problem as an efficient machine unlearning task with two stages. First, FUDLR identifies bias-inducing samples to unlearn through a novel bias-agnostic mask, optimized to balance fairness improvement with accuracy preservation. Its bias-agnostic design allows adaptability to various or co-existing biases simply by incorporating different fairness metrics. Second, FUDLR performs efficient debiasing by estimating and removing the influence of identified samples on model parameters. Extensive experiments demonstrate that FUDLR effectively and efficiently improves fairness while preserving recommendation accuracy, offering a practical path toward socially responsible LLM-RS. The code and data are available at https://github.com/JinLi-i/FUDLR.

Towards Fair Large Language Model-based Recommender Systems without Costly Retraining

TL;DR

This work tackles fairness in large language model based recommender systems by addressing two key obstacles: lack of generality across bias types and prohibitive retraining costs. It introduces FUDLR, a Fast Unified Debiasing framework that reinterprets debiasing as machine unlearning in two stages: (1) Bias Identification via a learnable mask that selects bias-inducing samples by optimizing a multi-objective loss balancing fairness, accuracy, and sparsity, and (2) Debiasing via Unlearning that uses influence functions to approximate a retrained model with a sparse subset removed, computing . The framework generalizes to multiple bias types through differentiable fairness metrics, leverages LoRA adapters and Hessian-vector products for efficiency, and achieves substantial fairness improvements with minimal accuracy loss and large speedups across real-world datasets. Extensive experiments on popularity and attribute biases demonstrate superior trade-offs between fairness and utility and significant reductions in computational cost compared with retraining-based baselines. The work offers a practical path toward socially responsible LLM-RS and opens avenues for refined, personalized debiasing in large-scale systems.

Abstract

Large Language Models (LLMs) have revolutionized Recommender Systems (RS) through advanced generative user modeling. However, LLM-based RS (LLM-RS) often inadvertently perpetuates bias present in the training data, leading to severe fairness issues. Addressing these fairness problems in LLM-RS faces two significant challenges. 1) Existing debiasing methods, designed for specific bias types, lack the generality to handle diverse or emerging biases in real-world applications. 2) Debiasing methods relying on retraining are computationally infeasible given the massive parameter scale of LLMs. To overcome these challenges, we propose FUDLR (Fast Unified Debiasing for LLM-RS). The core idea is to reformulate the debiasing problem as an efficient machine unlearning task with two stages. First, FUDLR identifies bias-inducing samples to unlearn through a novel bias-agnostic mask, optimized to balance fairness improvement with accuracy preservation. Its bias-agnostic design allows adaptability to various or co-existing biases simply by incorporating different fairness metrics. Second, FUDLR performs efficient debiasing by estimating and removing the influence of identified samples on model parameters. Extensive experiments demonstrate that FUDLR effectively and efficiently improves fairness while preserving recommendation accuracy, offering a practical path toward socially responsible LLM-RS. The code and data are available at https://github.com/JinLi-i/FUDLR.
Paper Structure (32 sections, 1 theorem, 19 equations, 4 figures, 9 tables)

This paper contains 32 sections, 1 theorem, 19 equations, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-based Recommender Systems
  4. Fairness in LLM-RS
  5. Problem Formulation
  6. The FUDLR Framework
  7. Bias Identification
  8. Debiasing via Unlearning
  9. Practical Discussion
  10. Generalizability to Various Bias Types
  11. Computational Efficiency
  12. Computational Complexity
  13. Experiments
  14. Experimental Setup
  15. Datasets
  16. ...and 17 more sections

Key Result

Proposition 1

Given a trained biased LLM-RS model parameterized by $\theta$ that minimizes the empirical risk on a training set $\mathcal{D}_{\rm train}$ with $n$ samples, and an identified subset of bias-inducing samples $\mathcal{D}_{\rm unlearn}$, the debiased parameter update $\Delta \theta$ required to appro where $\mathbf{H}_{\theta}$ is the invertible Hessian of the total training loss.

Figures (4)

  • Figure 1: Observations in the ML1M dataset. (a) Popularity bias: the backbone recommender (e.g., BIGRec DBLP:journals/tors/BaoZWZYLCFT25) exhibits a clear tendency to over-expose popular items while under-exposing long-tail items. Our FUDLR framework substantially alleviates this bias and produces a more balanced recommendation distribution aligned with true user preferences. (b) Attribute bias: our proposed FUDLR framework markedly improves fairness by reducing gender-related HR disparities, while the backbone model (e.g., BIGRec DBLP:journals/tors/BaoZWZYLCFT25) displays a significant performance gap between user groups.
  • Figure 2: The framework of FUDLR. For the fine-tuned LLM-RS model, it first identifies the bias-inducing training samples via a novel mask learning mechanism, which optimizes a flexible objective balancing fairness and accuracy. Then, it performs efficient debiasing by estimating and removing the influence of the identified samples on the model parameters.
  • Figure 3: Impacts of weighting parameters in FUDLR on popularity debiasing performance $F_{\rm pop}$ in the ML1M dataset.
  • Figure 4: Impacts of weighting parameters in FUDLR on attribute debiasing performance $F_{\rm attr}$ in the ML1M dataset.

Theorems & Definitions (1)

  • Proposition 1