LLM one-shot style transfer for Authorship Attribution and Verification

TL;DR

This work tackles unsupervised stylometry for authorship attribution and verification by introducing OSST, a framework that leverages log-probabilities from large language models to measure how well one text's style can be transferred to another via neutralization and one-shot demonstrations. The method avoids supervised data prone to topical biases and demonstrates favorable performance compared to prompting-based baselines and contrastive models, with gains that scale with model size and when using anchor-based verification. It also analyzes robustness under distribution shifts and shows multilingual strengths, providing a practical unsupervised approach for real-world authorship analysis. Limitations include reliance on in-distribution anchors and large base models, guiding future work on scaling laws and anchor selection for broader deployment.

Abstract

Computational stylometry studies writing style through quantitative textual patterns, enabling applications such as authorship attribution, identity linking, and plagiarism detection. Existing supervised and contrastive approaches often rely on datasets with spurious correlations, conflating style with topic. Despite the relevance of language modeling to these tasks, the pre-training of modern large language models (LLMs) has been underutilized in general authorship analysis. We introduce an unsupervised framework that uses the log-probabilities of an LLM to measure style transferability between two texts. This framework takes advantage of the extensive CLM pre-training and in-context capabilities of modern LLMs. Our approach avoids explicit supervision with spuriously correlated data. Our method substantially outperforms unsupervised prompting-based baselines at similar model sizes and exceeds contrastively trained models when controlling for topical overlap. Our framework's performance improves with model size. In the case of authorship verification, we present an additional mechanism that increases test-time computation to improve accuracy; enabling flexible trade-offs between computational cost and task performance.

Figures (10)

• Figure 1: High-level overview of OSST-based methods for authorship attribution and verification. Text highlighted in orange means that we obtain the log probabilities of the corresponding tokens, and {bracketed text} means variable interpolation. The top row defines common operations, while the bottom row defines the application of those operations to authorship verification and attribution.
• Figure 2: Closed-set authorship attribution results. Average accuracy and rank of correct candidate are reported for each model and dataset.
• Figure 4: F1 score on style change tasks in AV format.
• Figure 5: F1 score on authorship verification tasks.
• Figure 6: Average verification F1 across all datasets by number of anchor examples in nOSST. At $0$ we show the performance of zero-shot baseline, and at $10$ a single set of anchor samples was used.