KaiRacters: Character-level-based Writer Retrieval for Greek Papyri

TL;DR

This work tackles writer retrieval for Greek papyri by shifting from patch-based to character-based feature aggregation, focusing on the kai trigram and four additional letters. Using a NetVLAD-based WR pipeline with ResNet20 and NetRVLAD, trained in an unsupervised manner via SIFT-cluster pseudo-labels, the authors binarize pages with a U-Net and compute a global descriptor by aggregating character-level NetRVLAD features. They release a richly annotated GRK-120 dataset (including kai-s and four letters with quality labels), and demonstrate that aggregating about 11 kai-s per page can match or surpass the baseline using thousands of SIFT patches, achieving up to a $mAP$ improvement of about 4% (11% relative) on GRK-120. Qualitative analyses reveal complementary similarities between kai-s and SIFT patches, supporting the approach’s potential to aid paleographers in comparing writer styles. Overall, kai-s offer a more data-efficient and discriminative signal for WR in Greek papyri, with clear avenues for automatic character detection to reduce manual annotation.

Abstract

This paper presents a character-based approach for enhancing writer retrieval performance in the context of Greek papyri. Our contribution lies in introducing character-level annotations for frequently used characters, in our case the trigram kai and four additional letters (epsilon, kappa, mu, omega), in Greek texts. We use a state-of-the-art writer retrieval approach based on NetVLAD and compare a character-level-based feature aggregation method against the current default baseline of using small patches located at SIFT keypoint locations for building the page descriptors. We demonstrate that by using only about 15 characters per page, we are able to boost the performance up to 4% mAP (a relative improvement of 11%) on the GRK-120 dataset. Additionally, our qualitative analysis offers insights into the similarity scores of SIFT patches and specific characters. We publish the dataset with character-level annotations, including a quality label and our binarized images for further research.

Figures (6)

• Figure 1: Overview of our approach. Contrary to state of the art, we do not aggregate deep features of SIFT patches that usually contain a few strokes of handwriting. Instead, we aggregate features of specific characters, in our case the trigram kai, to form the global page descriptor.
• Figure 2: Distribution of the characters used per writer.
• Figure 3: Example page with annotated kai-s (green) in color and binarized. Note that we use a separate U-Net for binarizing characters and full pages (ID: Isak_5).
• Figure 4: Timeline of the documents included in GRK-120. The time of origin of some documents is still unknown.
• Figure 5: Similarity matrices per writer (pairwise document similarity averaged) when using (\ref{['fig:sim_sift']}) SIFT patches and (\ref{['fig:sim_kai']}) only kai-s for calculating the global page descriptor. The darker the square, the higher the similarity.