A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts

TL;DR

This work reframes authorship in the age of LLM paraphrasing through the Ship of Theseus lens, asking whether original authorship endures when texts are sequentially rewritten as $T^0 \to T^1 \to T^2 \to T^3$. It combines seven datasets, seven authors (six LLMs plus human) and four paraphrasers (including ChatGPT and PaLM2) to study traditional versus alternative ground-truth attributions and AI-detection under varying stylistic and content changes, using both a transparent stylometry pipeline and content embeddings. The key finding is that paraphrasing induces substantial style drift, especially for LLM paraphrasers, which shifts text toward the paraphraser’s own style and degrades attribution performance more than content similarity—yet adopting an alternative ground-truth where authorship follows the paraphraser yields substantially milder declines for many cases. The results highlight that ground truth in authorship and detection is task-dependent, with important implications for plagiarism policies, copyright disputes, and the design of robust AI-text detectors that must account for stylistic transformations introduced by paraphrasing tools. Overall, the study provides empirical and philosophical grounding for nuanced attribution frameworks in an era where paraphrasing and AI-generated writing increasingly blur the boundaries of authorship.

Abstract

In the realm of text manipulation and linguistic transformation, the question of authorship has been a subject of fascination and philosophical inquiry. Much like the Ship of Theseus paradox, which ponders whether a ship remains the same when each of its original planks is replaced, our research delves into an intriguing question: Does a text retain its original authorship when it undergoes numerous paraphrasing iterations? Specifically, since Large Language Models (LLMs) have demonstrated remarkable proficiency in both the generation of original content and the modification of human-authored texts, a pivotal question emerges concerning the determination of authorship in instances where LLMs or similar paraphrasing tools are employed to rephrase the text--i.e., whether authorship should be attributed to the original human author or the AI-powered tool. Therefore, we embark on a philosophical voyage through the seas of language and authorship to unravel this intricate puzzle. Using a computational approach, we discover that the diminishing performance in text classification models, with each successive paraphrasing iteration, is closely associated with the extent of deviation from the original author's style, thus provoking a reconsideration of the current notion of authorship.

Figures (11)

• Figure 1: Ship of Theseus paradox in text paraphrasing scenario: who should be considered the author of $T^n$?
• Figure 2: (A) indicates how LLMs can paraphrase as well as generate texts, (B) portrays the two alternative scenarios regarding authorship, and (C) shows how authorship should be determined.
• Figure 3: The original benchmark from li2023deepfake has several datasets with samples from different sources (human and LLMs) in each dataset. Original texts ($T^0$) are paraphrased sequentially three times, utilizing diverse paraphrasers (LLMs or PMs). We assess classifier performance in each iteration and measure their resemblance to $T^0$. For LLM paraphrasers, we additionally evaluate their similarity with text $G$, generated by the respective LLM.
• Figure 4: Confusion matrix for the Fine-tuned BERT classifier after the first version of paraphrasing (H: Human, C: ChatGPT, P: PaLM2, L: LLAMA, T: Tsinghua, E: Eleuther-AI, B: Bloom). In the case of LLM paraphrasers (ChatGPT and PaLM2), paraphrased samples are predominantly misclassified as corresponding LLMs, whereas for other PMs, such as Dipper (and Pegasus also), misclassifications are distributed more uniformly.
• Figure 5: Comparison of classification performance (avg. macro F1 score for Fine-tuned BERT), style deviation from the original, and content shift in successive paraphrased versions across various paraphrasing methods (averaged across all datasets and sources). Style and content deviations are calculated as the average of the cosine distance between the corresponding feature vector (author style model and text-embedding-ada-002 respectively) between paraphrased text $T^n$ and the original text $T^0$.