"Hi. I'm Molly, Your Virtual Interviewer!" -- Exploring the Impact of Race and Gender in AI-powered Virtual Interview Experiences

TL;DR

A comprehensive between-subjects study involving 218 participants across six distinct experimental conditions revealed that while the demographic attributes of the agents did not significantly influence the overall experience of interviewees, variations in the interviewees' demographics, significantly altered their perception of the AVI process.

Abstract

The persistent issue of human bias in recruitment processes poses a formidable challenge to achieving equitable hiring practices, particularly when influenced by demographic characteristics such as gender and race of both interviewers and candidates. Asynchronous Video Interviews (AVIs), powered by Artificial Intelligence (AI), have emerged as innovative tools aimed at streamlining the application screening process while potentially mitigating the impact of such biases. These AI-driven platforms present an opportunity to customize the demographic features of virtual interviewers to align with diverse applicant preferences, promising a more objective and fair evaluation. Despite their growing adoption, the implications of virtual interviewer identities on candidate experiences within AVIs remain underexplored. We aim to address this research and empirical gap in this paper. To this end, we carried out a comprehensive between-subjects study involving 218 participants across six distinct experimental conditions, manipulating the gender and skin color of an AI virtual interviewer agent. Our empirical analysis revealed that while the demographic attributes of the agents did not significantly influence the overall experience of interviewees, variations in the interviewees' demographics significantly altered their perception of the AVI process. Further, we uncovered that the mediating roles of Social Presence and Perception of the virtual interviewer critically affect interviewees' perceptions of fairness (+), privacy (-), and impression management (+).

Figures (4)

• Figure 1: Avatars of the virtual interviewer agent.
• Figure 2: Comparison of statistically significant group-wise differences of participant demographics (Black Female vs. White Female etc.) on key interview metrics: (a) Perceived Fairness (PF), (b) Social Presence and Perception (SPP), and (c) Impression Management (IM). Subfigures (a, b, c) display mean scores for different demographic groups, with error bars representing standard deviations.
• Figure 3: Effect of Participant Demographics on Social Presence and Perception (SPP): Coefficients represent the magnitude and direction of the impact each demographic has on SPP, highlighting how perceptions of social presence and perception vary across different groups. Each bar represents the effect size with corresponding 95% confidence intervals depicted through error bars. Significant findings are highlighted with a star($\star$).
• Figure 4: Direct and Indirect effects across user demographics under the mediation of Social Presence and Perception (SPP). Subplots (a), (b), and (c) illustrate the effects on Fairness Perception (FP), Privacy and Emotional Response (PER), and Impression Management (IM), respectively. Each bar represents the effect size with corresponding 95% confidence intervals depicted through error bars. Significant findings are highlighted with a star($\star$).