Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Challenging Negative Gender Stereotypes: A Study on the Effectiveness of Automated Counter-Stereotypes

Isar Nejadgholi, Kathleen C. Fraser, Anna Kerkhof, Svetlana Kiritchenko

TL;DR

Gender stereotypes persist in online discourse and can constrain opportunities and perceptions. The authors generate counter-stereotypes automatically using eleven strategies via ChatGPT and evaluate them with a 75-participant annotation study, yielding 185 usable counter-statements across ten strategies. The study identifies counter-facts and broadening universals as the most robust approaches, while humour and several other strategies tend to be ineffective or offensive, with variation more tied to stereotype target than rater gender. This work provides a publicly available dataset of ratings to guide future design of AI-assisted counter-speech, highlighting the need for audience-aware deployment, ethical considerations, and ongoing evaluation in online interventions.

Abstract

Gender stereotypes are pervasive beliefs about individuals based on their gender that play a significant role in shaping societal attitudes, behaviours, and even opportunities. Recognizing the negative implications of gender stereotypes, particularly in online communications, this study investigates eleven strategies to automatically counter-act and challenge these views. We present AI-generated gender-based counter-stereotypes to (self-identified) male and female study participants and ask them to assess their offensiveness, plausibility, and potential effectiveness. The strategies of counter-facts and broadening universals (i.e., stating that anyone can have a trait regardless of group membership) emerged as the most robust approaches, while humour, perspective-taking, counter-examples, and empathy for the speaker were perceived as less effective. Also, the differences in ratings were more pronounced for stereotypes about the different targets than between the genders of the raters. Alarmingly, many AI-generated counter-stereotypes were perceived as offensive and/or implausible. Our analysis and the collected dataset offer foundational insight into counter-stereotype generation, guiding future efforts to develop strategies that effectively challenge gender stereotypes in online interactions.

Challenging Negative Gender Stereotypes: A Study on the Effectiveness of Automated Counter-Stereotypes

TL;DR

Gender stereotypes persist in online discourse and can constrain opportunities and perceptions. The authors generate counter-stereotypes automatically using eleven strategies via ChatGPT and evaluate them with a 75-participant annotation study, yielding 185 usable counter-statements across ten strategies. The study identifies counter-facts and broadening universals as the most robust approaches, while humour and several other strategies tend to be ineffective or offensive, with variation more tied to stereotype target than rater gender. This work provides a publicly available dataset of ratings to guide future design of AI-assisted counter-speech, highlighting the need for audience-aware deployment, ethical considerations, and ongoing evaluation in online interventions.

Abstract

Gender stereotypes are pervasive beliefs about individuals based on their gender that play a significant role in shaping societal attitudes, behaviours, and even opportunities. Recognizing the negative implications of gender stereotypes, particularly in online communications, this study investigates eleven strategies to automatically counter-act and challenge these views. We present AI-generated gender-based counter-stereotypes to (self-identified) male and female study participants and ask them to assess their offensiveness, plausibility, and potential effectiveness. The strategies of counter-facts and broadening universals (i.e., stating that anyone can have a trait regardless of group membership) emerged as the most robust approaches, while humour, perspective-taking, counter-examples, and empathy for the speaker were perceived as less effective. Also, the differences in ratings were more pronounced for stereotypes about the different targets than between the genders of the raters. Alarmingly, many AI-generated counter-stereotypes were perceived as offensive and/or implausible. Our analysis and the collected dataset offer foundational insight into counter-stereotype generation, guiding future efforts to develop strategies that effectively challenge gender stereotypes in online interactions.
Paper Structure (17 sections, 3 figures, 3 tables)

This paper contains 17 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The Psychology of Stereotypes
  4. Countering Stereotypes with NLP
  5. Study Design
  6. Gender Stereotypes
  7. Generating Counter-Stereotypes
  8. Questionnaire
  9. Participants
  10. Results
  11. Offensiveness and Implausibility of Counter-Stereotypes
  12. Potential Effectiveness of Counter-Stereotypes
  13. Discussion
  14. Data Usability
  15. Conclusion
  16. ...and 2 more sections

Figures (3)

  • Figure 1: The description of the task presented to study participants.
  • Figure 2: Average ratings of potential effectiveness for the ten counter-strategies.
  • Figure 3: Average ratings of potential effectiveness broken down by the participants' gender and the group the stereotype is about (men/women). For example, "Men, stereotypes on men" refers to male participants' ratings for counter-stereotypes about men. Statistically significant differences between ratings of stereotypes about men and women are marked as * ($p<0.1$), ** ($p<0.05$), *** ($p<0.01$).