Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Identifying Gender Stereotypes and Biases in Automated Translation from English to Italian using Similarity Networks

Fatemeh Mohammadi, Marta Annamaria Tamborini, Paolo Ceravolo, Costanza Nardocci, Samira Maghool

TL;DR

The paper tackles gender bias in English-to-Italian machine translation by framing bias within legal and linguistic notions and addressing notional vs grammatical gender differences in Italian. It introduces a similarity-network approach using multilingual FastText embeddings and compares against Google Translate to quantify both the intensity and direction of bias for a broad word set. Two numerical features, GenderBiasIntensity and GenderBiasDirection, along with PostTranslationSimilarityChanges, enable fine-grained assessment of bias before and after translation, revealing embeddings as the primary source of bias and showing directional shifts after translation. The findings offer actionable guidance for developing gender-neutral MT, highlighting the need to debias training corpora and embeddings, and pointing to future work on alternative embeddings, multiple MT models, and expansion to other bias categories.

Abstract

This paper is a collaborative effort between Linguistics, Law, and Computer Science to evaluate stereotypes and biases in automated translation systems. We advocate gender-neutral translation as a means to promote gender inclusion and improve the objectivity of machine translation. Our approach focuses on identifying gender bias in English-to-Italian translations. First, we define gender bias following human rights law and linguistics literature. Then we proceed by identifying gender-specific terms such as she/lei and he/lui as key elements. We then evaluate the cosine similarity between these target terms and others in the dataset to reveal the model's perception of semantic relations. Using numerical features, we effectively evaluate the intensity and direction of the bias. Our findings provide tangible insights for developing and training gender-neutral translation algorithms.

Identifying Gender Stereotypes and Biases in Automated Translation from English to Italian using Similarity Networks

TL;DR

The paper tackles gender bias in English-to-Italian machine translation by framing bias within legal and linguistic notions and addressing notional vs grammatical gender differences in Italian. It introduces a similarity-network approach using multilingual FastText embeddings and compares against Google Translate to quantify both the intensity and direction of bias for a broad word set. Two numerical features, GenderBiasIntensity and GenderBiasDirection, along with PostTranslationSimilarityChanges, enable fine-grained assessment of bias before and after translation, revealing embeddings as the primary source of bias and showing directional shifts after translation. The findings offer actionable guidance for developing gender-neutral MT, highlighting the need to debias training corpora and embeddings, and pointing to future work on alternative embeddings, multiple MT models, and expansion to other bias categories.

Abstract

This paper is a collaborative effort between Linguistics, Law, and Computer Science to evaluate stereotypes and biases in automated translation systems. We advocate gender-neutral translation as a means to promote gender inclusion and improve the objectivity of machine translation. Our approach focuses on identifying gender bias in English-to-Italian translations. First, we define gender bias following human rights law and linguistics literature. Then we proceed by identifying gender-specific terms such as she/lei and he/lui as key elements. We then evaluate the cosine similarity between these target terms and others in the dataset to reveal the model's perception of semantic relations. Using numerical features, we effectively evaluate the intensity and direction of the bias. Our findings provide tangible insights for developing and training gender-neutral translation algorithms.

Paper Structure

This paper contains 19 sections, 4 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Gender Bias and Discrimination in Law and Linguistics
  4. Gender Bias and Discrimination in Computer Science
  5. Material and Methods
  6. Data collection
  7. MT reference
  8. Pre-processing and Word Embedding
  9. Similarity measurements
  10. Detecting gender bias
  11. Discussion
  12. Similarity Scores Analysis
  13. Similarity Scores in English Words
  14. Similarity Scores in Italian Words
  15. Gender Bias Detection and Analysis
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Scatter plot illustrating the distribution of similarity scores of English words relative to two different target words (he/she)
  • Figure 2: Scatter plot illustrating the distribution of similarity scores of Italian translations relative to two different target words (lui/lei)