Loci Similes: A Benchmark for Extracting Intertextualities in Latin Literature

TL;DR

Loci Similes introduces a Latin intertextuality benchmark comprising ~172k text segments and 545 expert-verified links to enable standardized evaluation of computational methods. The authors reframe intertextual detection as segment-level retrieval and alignment, and provide an evaluation framework with error-based metrics (SMR, FPR, FNR) and an extensible toolkit supporting three pipelines: retrieval, classification, and retrieve-and-rerank. Baseline experiments show that large multilingual dense retrievers excel at locating longer literal quotes and thematic allusions, while cross-encoders improve re-ranking; the combined retrieve-and-rerank pipeline reduces scholarly workload dramatically while recovering a substantial portion of true references. The work demonstrates the promise of language models for Latin intertextuality while identifying gaps in detecting subtle two-word reuses, laying groundwork for future architectures and expanded datasets with broader stylistic variation.

Abstract

Tracing connections between historical texts is an important part of intertextual research, enabling scholars to reconstruct the virtual library of a writer and identify the sources influencing their creative process. These intertextual links manifest in diverse forms, ranging from direct verbatim quotations to subtle allusions and paraphrases disguised by morphological variation. Language models offer a promising path forward due to their capability of capturing semantic similarity beyond lexical overlap. However, the development of new methods for this task is held back by the scarcity of standardized benchmarks and easy-to-use datasets. We address this gap by introducing Loci Similes, a benchmark for Latin intertextuality detection comprising of a curated dataset of ~172k text segments containing 545 expert-verified parallels linking Late Antique authors to a corpus of classical authors. Using this data, we establish baselines for retrieval and classification of intertextualities with state-of-the-art LLMs.

Figures (13)

• Figure 1: Example of intertextual reference. Reuse of a classic Vergilian phrase for speechlessness by Jerome. While retaining the semantic core, the author alters the word order to adapt the expression to a different context.
• Figure 2: Spectrum of intertextuality. References manifest in diverse forms, spanning from easily detectable verbatim quotations to adapted paraphrases and subtle allusions where only a semantic core remains.
• Figure 3: Distribution of confirmed references. Manually verified intertextual links in the annotated dataset by citing author (Jerome, Lactantius) and source author (including Virgil, Cicero, and others).
• Figure 4: Retrieve-and-rerank pipeline.Stage 1: The input text segment acts as a query to retrieve potential candidates from the database. Stage 2: To verify the reference, the query and source candidate are concatenated into a single input sequence to train a binary classifier.
• Figure 5: Performance vs. efficiency trade-off in the retrieve-and-rerank pipeline. The pipeline first generates $k$ candidates via embedding cosine similarity, followed by a binary classification stage to label pairs as Reference or No Reference. We compare this against a "Retrieval Only" baseline where the top $k$ candidates are treated as positive predictions, mimicking a scholar manually reviewing the top results. Results are averaged across 5 folds.