The Importance of Suppressing Domain Style in Authorship Analysis

TL;DR

The paper investigates how domain cues contaminate authorship analysis when using standard stylometric features like character trigrams. It introduces a domain-adversarial, domain-invariant style encoder and compares it to heuristic domain-suppression methods on a huge fanfiction-based corpus designed to test cross-domain attribution. The experiments show that trigram models suffer large performance drops under domain swapping (up to ~55 percentage points), while domain-adversarial training or heuristic masking keep losses to around 3–4 percentage points. The findings emphasize disentangling author style from domain style as essential for robust cross-domain authorship analysis, and suggest domain-adversarial approaches as scalable solutions, albeit with substantial data requirements.

Abstract

The prerequisite of many approaches to authorship analysis is a representation of writing style. But despite decades of research, it still remains unclear to what extent commonly used and widely accepted representations like character trigram frequencies actually represent an author's writing style, in contrast to more domain-specific style components or even topic. We address this shortcoming for the first time in a novel experimental setup of fixed authors but swapped domains between training and testing. With this setup, we reveal that approaches using character trigram features are highly susceptible to favor domain information when applied without attention to domains, suffering drops of up to 55.4 percentage points in classification accuracy under domain swapping. We further propose a new remedy based on domain-adversarial learning and compare it to ones from the literature based on heuristic rules. Both can work well, reducing accuracy losses under domain swapping to 3.6% and 3.9%, respectively.

Figures (1)

• Figure 1: Architecture of our network including writing style encoder, author classifier and domain classifier. The texts are fed into the network as $\mathbf{x}$, a recurrent neural network is used to process the sequential property of a text. The final hidden state is used as a fixed-size summarization of the sequence goodfellow:2016 for the following fully connected layers to yield the writing style representation $w(\mathbf{x})$ (with parameters $\theta_w$) of the full writing style encoder. The gradient can therefore only be obtained on the final hidden state which makes it difficult to optimize. This text representation is fed into the author classifier $c_a$ (parameters $\theta_a$) and domain classifier $c_d$ (parameters $\theta_d$). When backpropagating, we reverse the gradients for $\theta_d$ before the writing style encoder.