Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems

Jing Nathan Yan, Emma Harvey, Junxiong Wang, Jeffrey M. Rzeszotarski, Allison Koenecke

TL;DR

The RS practitioner workflow within large technology companies is mapped, focusing on how technical teams consider fairness internally and in collaboration with legal, data, and fairness teams, and key challenges to incorporating fairness into existing RS workflows are identified.

Abstract

Recommender systems (RS), which are widely deployed across high-stakes domains, are susceptible to biases that can cause large-scale societal impacts. Researchers have proposed methods to measure and mitigate such biases - but translating academic theory into practice is inherently challenging. Through a semi-structured interview study (N=11), we map the RS practitioner workflow within large technology companies, focusing on how technical teams consider fairness internally and in collaboration with legal, data, and fairness teams. We identify key challenges to incorporating fairness into existing RS workflows: defining fairness in RS contexts, balancing multi-stakeholder interests, and navigating dynamic environments. We also identify key organization-wide challenges: making time for fairness work and facilitating cross-team communication. Finally, we offer actionable recommendations for the RS community, including practitioners and HCI researchers.

Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems

TL;DR

The RS practitioner workflow within large technology companies is mapped, focusing on how technical teams consider fairness internally and in collaboration with legal, data, and fairness teams, and key challenges to incorporating fairness into existing RS workflows are identified.

Abstract

Recommender systems (RS), which are widely deployed across high-stakes domains, are susceptible to biases that can cause large-scale societal impacts. Researchers have proposed methods to measure and mitigate such biases - but translating academic theory into practice is inherently challenging. Through a semi-structured interview study (N=11), we map the RS practitioner workflow within large technology companies, focusing on how technical teams consider fairness internally and in collaboration with legal, data, and fairness teams. We identify key challenges to incorporating fairness into existing RS workflows: defining fairness in RS contexts, balancing multi-stakeholder interests, and navigating dynamic environments. We also identify key organization-wide challenges: making time for fairness work and facilitating cross-team communication. Finally, we offer actionable recommendations for the RS community, including practitioners and HCI researchers.

Paper Structure

This paper contains 55 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fairness in Machine Learning
  4. Fairness for Recommender Systems
  5. Fairness in Practice
  6. Bringing About Fairness Efforts
  7. Operationalizing Fairness
  8. Fairness Challenges in Practice
  9. Methods
  10. Recruitment and Participants
  11. Interviews
  12. Analysis
  13. Positionality
  14. The RS Practitioner Workflow
  15. An Overview of the RS Practitioner Workflow
  16. ...and 40 more sections

Figures (1)

  • Figure 1: Recommender System Practitioner Workflow. We visualize RS practitioners' interactions with the data team managing and monitoring production data access, the legal team auditing and approving specific data access requests, and the fairness (or responsible AI) team providing fairness expertise. Dotted borders indicate offline phases of the workflow, and solid borders indicate online phases. For each phase of the technical workflow (top row), there are associated phases within the fairness workflow (second row). In each phase, different participants interact (bottom row).