ALMs: Authorial Language Models for Authorship Attribution

TL;DR

This work tackles the challenge of authorship attribution by introducing Authorial Language Models (ALMs), a collection of author-specific causal language models fine-tuned on each candidate author's writings. The questioned document is attributed to the author whose corresponding ALM yields the lowest perplexity, with perplexity defined as the exponentiated average negative log-likelihood and computed via cross-entropy. On Blogs50, ALMs achieve a macro-average accuracy of 83.6%, while on CCAT50 they reach 74.9%, indicating strong performance and competitiveness with the state of the art. The method offers a token-level, flexible alternative to stylometry, and the results suggest robust attribution capabilities, especially for longer texts, with potential applications in forensics and automated linguistic analysis. Future directions include few-shot attribution via in-context learning LLMs and broader corpus-linguistic analyses using perplexity-based signals.

Abstract

In this paper, we introduce an authorship attribution method called Authorial Language Models (ALMs) that involves identifying the most likely author of a questioned document based on the perplexity of the questioned document calculated for a set of causal language models fine-tuned on the writings of a set of candidate author. We benchmarked ALMs against state-of-art-systems using the CCAT50 dataset and the Blogs50 datasets. We find that ALMs achieves a macro-average accuracy score of 83.6% on Blogs50, outperforming all other methods, and 74.9% on CCAT50, matching the performance of the best method. To assess the performance of ALMs on shorter texts, we also conducted text ablation testing. We found that to reach a macro-average accuracy of 70%, ALMs needs 40 tokens on Blogs50 and 400 tokens on CCAT50, while to reach 60% ALMs requires 20 tokens on Blogs50 and 70 tokens on CCAT50.

Figures (1)

• Figure 1: Performance of ALMs Under Different Test Text Lengths