Leveraging Structure Knowledge and Deep Models for the Detection of Abnormal Handwritten Text

TL;DR

A two-stage detection algorithm that combines structure knowledge and deep models for the destruction of the sequence structure in handwritten text and can greatly improve the detection performance.

Abstract

Currently, the destruction of the sequence structure in handwritten text has become one of the main bottlenecks restricting the recognition task. The typical situations include additional specific markers (the text swapping modification) and the text overlap caused by character modifications like deletion, replacement, and insertion. In this paper, we propose a two-stage detection algorithm that combines structure knowledge and deep models for the above mentioned text. Firstly, different structure prototypes are roughly located from handwritten text images. Based on the detection results of the first stage, in the second stage, we adopt different strategies. Specifically, a shape regression network trained by a novel semi-supervised contrast training strategy is introduced and the positional relationship between the characters is fully employed. Experiments on two handwritten text datasets show that the proposed method can greatly improve the detection performance. The new dataset is available at https://github.com/Wukong90.

Figures (11)

• Figure 1: Some typical modifications are effectively detected by the proposed method.
• Figure 2: According to the various distinct shapes, we can define structure prototypes. The same overlap prototype may correspond to different modifications. In our experiments, the English dataset includes types I, II and the Chinese dataset includes types I-IV.
• Figure 3: The morphological characteristics of the ideal markers.
• Figure 4: The proposed two data augmentation methods, the scale expansion, and the dynamic location change are illustrated.
• Figure 5: The pipeline of the proposed two-stage detection algorithm for the abnormal text.