Can Interpretability Layouts Influence Human Perception of Offensive Sentences?

TL;DR

This paper investigates whether three ML interpretability layouts influence how people judge hate-speech sentences, focusing on Misogyny and Racism. It employs Integrated Gradients to generate explanations and Generalized Additive Models to analyze responses from 114 participants across 3 layouts rating 20 sentences on a 7-point scale. The statistical results show no significant effect of the layouts on participants' classifications, though qualitative feedback highlights interpretability's value for triggering corrective feedback and gaining deeper insights into model behavior beyond standard metrics. The findings contribute empirical evidence about interpretability in online communities and motivate future work studying different norms and domains beyond hate speech.

Abstract

This paper conducts a user study to assess whether three machine learning (ML) interpretability layouts can influence participants' views when evaluating sentences containing hate speech, focusing on the "Misogyny" and "Racism" classes. Given the existence of divergent conclusions in the literature, we provide empirical evidence on using ML interpretability in online communities through statistical and qualitative analyses of questionnaire responses. The Generalized Additive Model estimates participants' ratings, incorporating within-subject and between-subject designs. While our statistical analysis indicates that none of the interpretability layouts significantly influences participants' views, our qualitative analysis demonstrates the advantages of ML interpretability: 1) triggering participants to provide corrective feedback in case of discrepancies between their views and the model, and 2) providing insights to evaluate a model's behavior beyond traditional performance metrics.

Figures (1)

• Figure 1: The local interpretability and sum of relevance scores layouts. Figure \ref{['fig:UserStudyLocalInterpretability']} represents the interpretability of classifying a sentence with the "Misogyny" class, while Figure \ref{['fig:UserStudySumRelevance']} illustrates the sum of relevance scores for the "Misogyny" class considering words across all instances in the training dataset. In Figure \ref{['fig:UserStudyLocalInterpretability']}, the intensity of the green shade indicates the relevance of the highlighted word to "Misogyny," demonstrating the contribution of those words to identifying the text as misogynistic. In contrast, the intensity of the red shade is related to the decrease in the violation confidence, demonstrating the contribution of those words to classifying the text as not misogynistic.