Character Recognition in Byzantine Seals with Deep Neural Networks

TL;DR

The paper addresses automatic reading of text on Byzantine seals by proposing a two-stage deep CNN pipeline: a first network localizes characters (achieving $mAP@0.5 > 0.9$) and a second network classifies the localized characters (achieving $Top-1$ accuracy $> 0.92$), followed by post-processing to yield a diplomatic transcription. End-to-end evaluation demonstrates the approach's efficiency relative to state-of-the-art methods for similar tasks. The results enable automated transcription of inscriptions on Byzantine seals, providing scalable access to historical text sources and facilitating epigraphic and prosopographic studies.

Abstract

Seals are small coin-shaped artifacts, mostly made of lead, held with strings to seal letters. This work presents the first attempt towards automatic reading of text on Byzantine seal images.Byzantine seals are generally decorated with iconography on the obverse side and Greek text on the reverse side. Text may include the sender's name, position in the Byzantine aristocracy, and elements of prayers. Both text and iconography are precious literary sources that wait to be exploited electronically, so the development of computerized systems for interpreting seals images is of paramount importance. This work's contribution is hence a deep, two-stages, character reading pipeline for transcribing Byzantine seal images. A first deep convolutional neural network (CNN) detects characters in the seal (character localization). A second convolutional network reads the localized characters (character classification). Finally, a diplomatic transcription of the seal is provided by post-processing the two network outputs. We provide an experimental evaluation of each CNN in isolation and both CNNs in combination. All performances are evaluated by cross-validation. Character localization achieves a mean average precision (mAP@0.5) greater than 0.9. Classification of characters cropped from ground truth bounding boxes achieves Top-1 accuracy greater than 0.92. End-to-end evaluation shows the efficiency of the proposed approach when compared to the SoTA for similar tasks.