Explainability of machine learning approaches in forensic linguistics: a case study in geolinguistic authorship profiling

TL;DR

The paper tackles the challenge of explainable ML in forensic geolinguistic profiling by applying a post-hoc leave-one-word-out framework to transformer-based dialect classifiers trained on a large German social media corpus. Across 3-, 4-, and 5-class geographic schemes, the approach yields strong accuracies and reveals that many influential lexical items are place names or region-specific forms, offering interpretable evidence for predictions. The findings demonstrate that interpretable features can support forensic reasoning and verification, while also highlighting data noise and the need for preprocessing steps to mitigate non-dialectal signals. Overall, the work advances practical explainability in forensic linguistics and outlines concrete directions for improving data quality and robustness in future studies.

Abstract

Forensic authorship profiling uses linguistic markers to infer characteristics about an author of a text. This task is paralleled in dialect classification, where a prediction is made about the linguistic variety of a text based on the text itself. While there have been significant advances in recent years in variety classification, forensic linguistics rarely relies on these approaches due to their lack of transparency, among other reasons. In this paper we therefore explore the explainability of machine learning approaches considering the forensic context. We focus on variety classification as a means of geolinguistic profiling of unknown texts based on social media data from the German-speaking area. For this, we identify the lexical items that are the most impactful for the variety classification. We find that the extracted lexical features are indeed representative of their respective varieties and note that the trained models also rely on place names for classifications.

Figures (1)

• Figure 1: Operationalization of dialect regions