What Does a Software Engineer Look Like? Exploring Societal Stereotypes in LLMs
Muneera Bano, Hashini Gunatilake, Rashina Hoda
TL;DR
This work analyzes how two large language models, GPT-4 and Microsoft Copilot, reproduce gender and racial stereotypes in software engineering recruitment through both text and image outputs. Using a controlled recruitment scenario, the authors generate 300 synthetic profiles (100 gender-based and 50 gender-neutral per model) and evaluate top-5 candidate selections and best candidates against job requirements across four SE roles, complemented by LLM-generated candidate images. The findings show consistent biases toward male and Caucasian profiles, especially for senior roles, and image biases toward lighter skin, lean body types, and younger ages, with model-specific patterns such as Copilot's positional bias in selecting the best candidate. The study provides a replicable methodology and bias-analysis framework, highlighting the need for auditable, inclusive AI deployment in SE and offering concrete recommendations to mitigate discriminatory outcomes in AI-assisted recruitment and beyond.
Abstract
Large language models (LLMs) have rapidly gained popularity and are being embedded into professional applications due to their capabilities in generating human-like content. However, unquestioned reliance on their outputs and recommendations can be problematic as LLMs can reinforce societal biases and stereotypes. This study investigates how LLMs, specifically OpenAI's GPT-4 and Microsoft Copilot, can reinforce gender and racial stereotypes within the software engineering (SE) profession through both textual and graphical outputs. We used each LLM to generate 300 profiles, consisting of 100 gender-based and 50 gender-neutral profiles, for a recruitment scenario in SE roles. Recommendations were generated for each profile and evaluated against the job requirements for four distinct SE positions. Each LLM was asked to select the top 5 candidates and subsequently the best candidate for each role. Each LLM was also asked to generate images for the top 5 candidates, providing a dataset for analysing potential biases in both text-based selections and visual representations. Our analysis reveals that both models preferred male and Caucasian profiles, particularly for senior roles, and favoured images featuring traits such as lighter skin tones, slimmer body types, and younger appearances. These findings highlight underlying societal biases influence the outputs of LLMs, contributing to narrow, exclusionary stereotypes that can further limit diversity and perpetuate inequities in the SE field. As LLMs are increasingly adopted within SE research and professional practices, awareness of these biases is crucial to prevent the reinforcement of discriminatory norms and to ensure that AI tools are leveraged to promote an inclusive and equitable engineering culture rather than hinder it.