Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages

My H. Dinh, James Kotary, Ferdinando Fioretto

TL;DR

This paper shows how efficiently-solvable fair ranking models, based on the optimization of Ordered Weighted Average (OWA) functions, can be integrated into the training loop of an LTR model to achieve favorable balances between fairness, user utility, and runtime efficiency.

Abstract

Learning to Rank (LTR) is one of the most widely used machine learning applications. It is a key component in platforms with profound societal impacts, including job search, healthcare information retrieval, and social media content feeds. Conventional LTR models have been shown to produce biases results, stimulating a discourse on how to address the disparities introduced by ranking systems that solely prioritize user relevance. However, while several models of fair learning to rank have been proposed, they suffer from deficiencies either in accuracy or efficiency, thus limiting their applicability to real-world ranking platforms. This paper shows how efficiently-solvable fair ranking models, based on the optimization of Ordered Weighted Average (OWA) functions, can be integrated into the training loop of an LTR model to achieve favorable balances between fairness, user utility, and runtime efficiency. In particular, this paper is the first to show how to backpropagate through constrained optimizations of OWA objectives, enabling their use in integrated prediction and decision models.

Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages

TL;DR

This paper shows how efficiently-solvable fair ranking models, based on the optimization of Ordered Weighted Average (OWA) functions, can be integrated into the training loop of an LTR model to achieve favorable balances between fairness, user utility, and runtime efficiency.

Abstract

Learning to Rank (LTR) is one of the most widely used machine learning applications. It is a key component in platforms with profound societal impacts, including job search, healthcare information retrieval, and social media content feeds. Conventional LTR models have been shown to produce biases results, stimulating a discourse on how to address the disparities introduced by ranking systems that solely prioritize user relevance. However, while several models of fair learning to rank have been proposed, they suffer from deficiencies either in accuracy or efficiency, thus limiting their applicability to real-world ranking platforms. This paper shows how efficiently-solvable fair ranking models, based on the optimization of Ordered Weighted Average (OWA) functions, can be integrated into the training loop of an LTR model to achieve favorable balances between fairness, user utility, and runtime efficiency. In particular, this paper is the first to show how to backpropagate through constrained optimizations of OWA objectives, enabling their use in integrated prediction and decision models.
Paper Structure (23 sections, 25 equations, 8 figures, 2 tables, 1 algorithm)

This paper contains 23 sections, 25 equations, 8 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Setting and Goals
  4. Fairness of Exposure
  5. Limitations of Fair LTR Methods
  6. Smart OWA Optimization for Fair Learning to Rank (SOFaiR)
  7. Ordered Weighted Averaging Operator
  8. End-to-End Learning in SOFaiR
  9. Forward Pass Optimization
  10. Backpropagation
  11. Experiments
  12. Running Time Analysis
  13. Fairness and Utility Tradeoffs Analysis
  14. Multi-Group Fairness Analysis
  15. Related Work
  16. ...and 8 more sections

Figures (8)

  • Figure 1: The differentiable optimization module proposed in SOFaiR. Its forward pass is calculated by an efficient Frank-Wolfe method, and its backward pass computes the SPO+ subgradient of the OWA problem's regret due to prediction error.
  • Figure 2: Yahoo-20: Fairness violation at query level.
  • Figure 3: The differentiable optimization module employed in SOFaiR. It forward pass solves the problem \ref{['model:OWA_fair_rank']} by an efficent Frank-Wolfe method. Its backward pass calculates the SPO+ subgradient, relative to its equivalent, but intractably large LP form.
  • Figure 4: Running time benchmark on MSLR-Web10k dataset
  • Figure 5: Benchmarking performance in term of fairnesss-utility trade-off on Yahoo-20 (top left), and Yahoo-40(top right). MSLR-20(bottom-left), MSLR-100 (bottom-right)
  • ...and 3 more figures