Practitioner Insights on Fairness Requirements in the AI Development Life Cycle: An Interview Study

TL;DR

<3-5 sentence high-level summary> The paper investigates how AI/ML practitioners understand and operationalize fairness across the software development lifecycle. Through 26 semi-structured interviews across 23 countries, it analyzes how fairness concerns are translated into requirements, documented, and validated, and how early in the SDLC these concerns arise. The findings reveal that while awareness of fairness concepts exists, definitions and practices are highly inconsistent, with data quality and representativeness repeatedly cited as core issues and formal documentation often lacking. The study argues for context-aware fairness definitions, standardized processes, and stakeholder-inclusive governance to embed fairness into AI projects from requirements elicitation through deployment and maintenance.</3-5 sentence high-level summary>

Abstract

Nowadays, Artificial Intelligence (AI), particularly Machine Learning (ML) and Large Language Models (LLMs), is widely applied across various contexts. However, the corresponding models often operate as black boxes, leading them to unintentionally act unfairly towards different demographic groups. This has led to a growing focus on fairness in AI software recently, alongside the traditional focus on the effectiveness of AI models. Through 26 semi-structured interviews with practitioners from different application domains and with varied backgrounds across 23 countries, we conducted research on fairness requirements in AI from software engineering perspective. Our study assesses the participants' awareness of fairness in AI / ML software and its application within the Software Development Life Cycle (SDLC), from translating fairness concerns into requirements to assessing their arising early in the SDLC. It also examines fairness through the key assessment dimensions of implementation, validation, evaluation, and how it is balanced with trade-offs involving other priorities, such as addressing all the software functionalities and meeting critical delivery deadlines. Findings of our thematic qualitative analysis show that while our participants recognize the aforementioned AI fairness dimensions, practices are inconsistent, and fairness is often deprioritized with noticeable knowledge gaps. This highlights the need for agreement with relevant stakeholders on well-defined, contextually appropriate fairness definitions, the corresponding evaluation metrics, and formalized processes to better integrate fairness into AI/ML projects.